Edward O. Pyzer-Knapp - Deep Learning for Physical Scientists

1 A fundamental problem, which is impeding progress;

2 A solution to that problem (often theoretical); and crucially

3 The capability to fully realise that solution.

Whilst this may seem obvious, the lack of (3) can have dramatic consequences. Imperfectly realised solutions, especially if coupled with overzealous marketing (or hype) can set back a research field significantly – sometimes resulting in decades in the wilderness.

Machine learning has suffered this fate not once, but twice – entering the so‐called AI‐winters where only a few brave souls continued to work. The struggle was not in vain, however, and breakthroughs in the theory – especially the rapid improvement and scaling of deep learning – coupled with strides in computational capability have meant that the machine learning age seems to be upon us with a vengeance.

This “era” of AI feels more permanent, as we are finally seeing successful applications of AI in areas where it had previously struggled. Part of this is due to the wide range of tools which are at the fingertips of the everyday researcher, and part of it is the willingness of people to change the way they think about problems so as to use these new capabilities to their full potential.

The aims of this book are twofold:

1 Introduce you to the prevalent techniques in deep learning, so that you can make educated decisions about what the best tools are for the job.

2 Teach you to think in such a way that you can come up with creative solutions to problems using the tools that you learn throughout this book.

Typically in the sciences, particularly the physical sciences, we focus on the model rather than on the task that it is providing. This can lead to siloing of techniques in which (relatively) minor differences become seemingly major differentiating factors.

As a data‐driven researcher it becomes much more important to understand broad categorisations and find the right tools for the job, to be skilled at creating analogies between tasks, and to be OK with losing some explainability of the model (this is particularly true with deep learning) at the gain of enabling new capabilities. More than any other area I have been involved with, deep learning is more a philosophy than any single technique; you must learn to “think deep.” Will deep learning always be the right approach? Of course not, but with growing experience you will be able to intuit when to use these tools, and also be able to conceive of new ways in which these tools can be applied. As we progress through this journey I will show you particularly innovative applications of deep learning, such as the neural fingerprint, and task you to stretch your imagination through the use of case studies. Who knows, you might even improve upon the state of the art!

1 Never get tied into domain specific language, instead use broad terms which describe the process you are trying to achieve.

2 Look for analogies between tasks. By forcing yourself to “translate” what is going on in a particular process you often come to understand more deeply what you are trying to achieve.

3 Understand the difference between data and information. A few well chosen pieces of data can be more useful than a ton of similar data points.

4 Finally, never lose sight of your true goal – at the end of the day this is more likely to be “provide insight into process X” than “develop a super fancy method for predicting X.” The fancy method may be necessary, but it is not the goal.

Thinking back to the three ingredients of a paradigm shift, we remember that one of the major blockers to achieving this is the lack of a capability to implement the solution we have dreamed up. Therefore, throughout this book, I will be teaching you how to use a state of the art deep‐learning framework known as TensorFlow and provide real world examples. These examples will not be aimed at squeezing every last piece of performance out of the system, but instead at ensuring that you understand what is going on. Feel free to take these snippets and tune them yourself so that they work well for the problems you are tackling. Finally, I hope that you get as much fun out of coming on this journey with me, as I have had putting it together. I hope that this book inspires you to start breaking down barriers and drive innovation with data not just in your domain, but in everything you do.

Why use python? There are a lot of programming languages out there – and they all have their plus and minuses. In this book, we have chosen to use Python as our language of choice. Why is this?

First of all, is the ease of understanding. Python is sometimes known as “executable pseudo code,” which is a reference to how easy it is to write basic code. Now this is obviously a slight exaggeration (and it is very possible to write illegible code in Python!), but Python does represent a good trade‐off between compactness and legibility. There is a philosophy which went into developing Python which states “There should be one (and preferably only one) obvious way to do a task.” To give you an illustrative example, here is how you print a string in Python:

print("Hello World!")

It is clear what is going on! In Java it is a little more obscure, to deal with system dependencies:

system.out.println("Hello World!")

And in C, it is not obvious at all what is going on (C is a compiled language so it really only needs to tell the compiler what it needs to do):

“Hello World!" >> cout

In fact, C code can be so hard to read that there is actually a regular competition to write obfuscated C code, so unreadable it is impossible to work out what is going on – take a look at https://www.ioccc.org/and wonder at the ingenuity. So by choosing to use Python in this book even if you are not a regular Python user you should be able to have a good understanding of what is going on.

Second is the transferability. Python is an interpreted language, and you do not need to compile it into binary in order to run it. This means that whether you run on a Mac, Windows, or Linux machine, so long as you have the required packages installed you do not have to go through any special steps to make the code you write on one machine run on another. I recommend the use of a Python distribution known as Anaconda to take this to a new level, allowing very fast and simple package installation which takes care of package dependencies. Later on, in this chapter, we will step through installing Anaconda and setting up your Python environment.

One other reason for using Python is the strong community, which has resulted in a huge amount of online support for those getting into the language. If you have a problem when writing some code for this book, online resources such as stackoverflow.comare full of people answering questions for people who have had the exact same problem. This community has resulted in the surfacing of common complaints, and the community collectively building solutions to make libraries for solving these problems and to deliver new functionality. The libraries publically available for Python are something quite special, and are one of the major reasons it has become a major player in the data science and machine learning communities.

Recently, Python has seen a strong emergence in the data science community, challenging more traditional players such as R and Matlab. Aside from the very intuitive coding style, transferability, and other features described above, there are a number of reasons for this. First amongst these is its strong set of packages aimed at making mathematical analysis easy. In the mid‐1990s the Python community strongly supported the development of a package known as numeric whose purpose was to take the strengths of Matlab's mathematical analysis packages and bring them over to the Python ecosystem. Numeric evolved into numpy, which is one of the most heavily used Python packages today. The same approach was taken to build matplotlib – which as the name suggests was built to take the Matlab plotting library over to python. These were bundled with other libraries aimed at scientific applications (such as optimisation) and turned into scipy – Python's premier scientific‐orientated package.