Robert J. Moffat - Planning and Executing Credible Experiments

Professor Henk’s experience includes industry, government labs, and academia. His work included wind tunnel tests at Virginia Tech and Japan’s Mach 5 tunnel at NAL, and water tunnels at the U.S. Naval Research Lab and at Stanford. He worked internationally in the aerospace industry, designing and testing advanced engine components with IHI ( ). He spearheaded im‐provements to the turbine engine design process. A registered Professional Engineer, Dr. Henk is keenly interested in appropriate energy.

Dr. Henk’s recent work focused on experimental methods. He has done diverse experiments, from field work on environmental flows, to tests in pristine laboratories, to materials tests for structures. He founded Royal HanMi to promote international energy science and design of experiments.

Professor Henk has taught at universities, public and private, internationally as well as in the USA. Two of Dr. Henk’s courses, Experiment Design and Statistical Modeling, drew students from the medical school, business, engineering and environmental schools. Researchers learned advanced strategies to select data and how to draw strong defensible conclusions from data.

Together with colleagues at Kyoto University ( ), Science University of Tokyo, IHI, and Tokyo University they helped the nation brave the impact of the Fukushima Tsunami. Together with colleagues at LeTourneau, Virginia Military Institute, Handong ( ) University, he trained our future. Dr. Henk has served within the Commonwealth of Virginia’s Governor’s Schools and STEM Academies.

Dr. Henk’s youth science book, UnLock Rocks, explores how rocks and crystals reveal the age of the Earth. Simple experiments, using kitchen ingredients, make concepts tangible and tasty. Two books were published in South Korea.

Titles

UnLock Rocks

Physics 1 Lab with Experiment Design (RoK)

Physics 2 Lab with Experiment Design (RoK)

with R.J. Moffat

Planning and Executing Credible Experiments

Laboratories (and businesses) need people who can answer difficult questions experimentally. If data are needed for decision‐making, this book is for you.

Science, medicine, the environment, agriculture, and engineering depend heavily on experiments. Business does as well, when surveying the added complexity of human taste and markets. In any of these fields, an experiment must be credible or it wastes time and resources. This book offers a tested guide, developed and used for decades at Stanford University, to equip novice researchers and experienced technicians to plan and execute credible experiments.

This book prepares the reader to anticipate the choices one will face from the launch of your project to the final report. We come alongside the reader, emphasizing the strategies of experiment planning, execution, and reporting.

The foundation of this book originated in “Planning Experimental Programs,” a set of class notes by Robert Moffat, our first author. Our second author, Roy Henk, is one of thousands whose own experiments benefited from Moffat's notes. This book blends our approaches to developing and planning experimental programs.

Bob Moffat : Our goal for this book was to collect, organize, summarize, and present what we think are important ideas about developing, running, and reporting reproducible, provably accurate experiments – with minimum wasted effort.

My contribution has to do with the process of developing experimental programs that can be trusted to produce trustworthy data. I used the word “developing” because “new” experiments generally require quite a few changes to eliminate unforeseen (but finally recognized) errors before they qualify as “good” experiments. This is called “the shakedown phase.”

This raises a question, however: how can you spot an error in a “new” experiment? The only way I know is to run a test for which you already know the answer. We have the three conservation laws we can trust (conservation of mass, momentum, and energy), but they are a bit hard to work with. I think it generally requires less work to simply stretch the operable domain of the proposed experiment until it includes some conditions that have already yielded data that you trust. That data become the “qualification test” for your experiment.

I've been designing and running experiments for more than 60 years, and my experience has been that the development of a good experiment is always iterative. It takes a lot of work during the shakedown period to eliminate all the sources of error. Surprisingly, this aspect of experiment planning isn't generally “taught.” That's really what started this book! The suggestions presented here reflect things I learned “on the job” during my first 10 years in industry and then refined during the rest of my career at Stanford.

When I got my BSME from the University of Michigan, I was offered a job in the General Motors (GM) Research Laboratories (GMR). I stayed with GMR almost exactly 10 years, rising from working on little problems like calibrating instruments to developing a high‐precision, high‐temperature wind tunnel for testing aircraft engine thermocouple probes up to 1600 °F, finding what caused the bearings to fail on an experimental engine, and developing high‐effectiveness heat exchangers for gas turbine engines. In the last two years of my stay, I was part of a small group of engineers who designed, built, and tested a high‐efficiency, two‐spool regenerative gas turbine that, starting from 30° below zero Fahrenheit, could deliver 350 HP in less than two minutes. That was developed for GM trucks and buses, but a descendant of that engine powered the US Army's M1A1 battle tank – the one designed for cold weather.

The increasingly interesting problems I encountered in the last three or four years made it clear to me that I needed to go back to school – there was too much that I didn't know.

I applied to Stanford with references from GMR and was offered a scholarship, which lead to a PhD, a Stanford professorship, 15 years as head of the mechanical engineering department, and 25 years of research and teaching on engineering problems. In my spare time, I worked with Dr. Alvin Hackel, a Stanford pediatrician, to develop the Stanford Transport Incubator. Dr. Hackel provided the medical insights and I did the engineering. His patients, usually newborn and sometimes premature, often needed intensive medical care during transport between hospitals. That incubator won the American Society of Mechanical Engineers' 1987 Holley Medal for Service for the Benefit of Mankind and is displayed on the cover.

I retired and began to put this book together.

Now a word from my colleague, Roy Henk.

Roy Henk : With the aim to continually improve, experiments can fill one's personal daily life. My experimenting began on a small farm with vegetables and livestock (plus honeybees). Later experiments involved concrete mixing, internal combustion engines, molten metal, and measuring ore. At Virginia Tech, I joined low‐noise precision wind‐tunnel tests on flow transition; I have used wind tunnels at Stanford and in Japan at the National Aerospace Laboratory of Japan (NAL), even joining tests in a Mach 5 tunnel. NAL together with JAXA is Japan’s equivalent to NASA. My research used water tunnels at the US Naval Research Lab and at Stanford, and Bob’s notes. Ten years in industry had me designing advanced components of commercial aerospace engines. Compared to air flow around an airplane, thermo‐fluid flow inside an aerospace engine is about the most challenging area of classical physics. Those engines have flown on aircraft for years without incident.