I spent four years inside Tesla's neural nets trying to map pixels to steering angles. Then I spent another chunk inside LLMs trying to map tokens to prose. Both taught me the same thing: the biggest breakthroughs in AI-in-science won't come from the models. They'll come from how you frame the problem. Here's a step-by-step tutorial for anyone who wants to actually use neural nets to advance scientific research — not just feed a CSV into a Jupyter notebook and pray. I'm going to use a concrete example: predicting protein-ligand binding affinity from sequence. But the steps apply to any biological or physical system where you have data and a question. Step 1: Shrink the question until it hurts Don't start with "Let's predict all drug interactions." Start with "Can I predict whether this specific mutation changes binding on this one protein class?" At Tesla we'd train a tiny model on a single intersection before scaling. deeplearning-for-biology is full of papers that blew a million dollars on compute only to discover the problem was ill-posed. Practical tip: Write down the input and output as tensors. Input: (batch, sequence_length, 20) for one-hot amino acids. Output: (batch, 1) for binding energy. If you can't write the tensor shapes, you don't yet have a problem. Step 2: Curate the data like a mad scientist The internet is total garbage for science. You can't scrape PubMed abstracts and call it a dataset. The most powerful trick I know: sort your dataset descending by loss after a quick 20-minute training run. You will find label errors, wrong structures, and experimental noise hiding in the bottom 10%. Fix those. Fixing labels often beats adding more data. Common pitfall: using noisy public benchmarks (PDBbind, etc.) without cleaning. Half the datapoints are borderline wrong. One time we found that 30% of the binding affinities in a "standard" set were mismatched with the crystal structures. We wasted three weeks chasing a phantom. Step 3: Build a feedforward baseline before you touch a transformer I cannot emphasize this enough. transformers-in-research are seductive because they work on everything, but they also hide your mistakes. Before any attention, implement a simple MLP on fixed-length features. If that baseline doesn't capture signal, your data pipeline is broken. At that point it's not a model problem. Practical tip: Use 2 hidden layers of 256 units, ReLU, Adam lr 3e-4. Train for 10 minutes on CPU. If the training loss doesn't go down at all, back to Step 2. Step 4: Graduate to a transformer — but keep it tiny Once the baseline works, build a small transformer. 4 layers, 8 heads, 128-d embedding. Don't use the fancy rotary embeddings yet. Just absolute positional encoding with causal masking (even if your binding isn't casual, it helps regularize). Train it. It should outperform the MLP by at least 10% on validation. If it doesn't, you're either underfitting (too small) or the attention isn't finding any structure. Common pitfall: using full attention on long sequences. Many proteins are 500+ residues; O(n²) kills you. Use Performer or linear attention (I recommend the random feature method). It's ugly but it works. Step 5: Inject physical inductive biases (this is where the magic happens) Neural nets don't know physics. They think a hydrophobic pocket and a hydrophilic patch are interchangeable. So give them hints. Encode calculated features like isoelectric point, hydrophobicity scale, or even a simple distance matrix from a predicted structure. I like to concatenate these as auxiliary tokens or add them as extra channels in the embedding. One thing I've seen work well: pretrain on masked language modeling (MLM) over protein sequences from UniRef, then fine-tune on your binding task. The AI-in-science community calls this "foundation model for biology". It's just transfer learning. Don't overcomplicate it. Step 6: Sweep the learning rate, then sweep nothing else The most important hyperparameter is learning rate. For small transformers on protein data, 1e-4 with cosine decay and 10% warmup usually works. Batch size: as large as your GPU memory allows (but not 1). Weight decay: 0.1. That's it. Don't touch the number of heads or layers until you've done the LR sweep. Common pitfall: tuning architecture before tuning data quality. It's always the data. Step 7: Trust the tail, not the average Science is about the hard cases. When you evaluate your model, don't report R² over the whole validation set. Look at the worst 5% predictions. Those are the mutations that stump your model. Those are the real discovery opportunities. I have a rule: if your model's performance on the hardest 5% is random, your model is useless for science. A scientist doesn't care that the average is good; they care about the one outlier that might be a new drug target. --- The hardest part of deeplearning-for-biology isn't the coding. It's the patience to clean data, the humility to start small, and the honesty to admit when the model is just memorizing noise. I spent six months on one project where the neural net was doing a beautiful job... of predicting the lab's batch effects. Not biology. Lost time. Learn from my pain. Go build something that actually helps a biologist. And when your transformer spits out a prediction that contradicts the literature? Trust it. Then go run the experiment. That's the game.