\emph{Neural surrogates}, neural networks that are trained to compute the same function as a given program, provide alternative representations of classical programs with different properties that are useful for a variety of programming tasks. With neural compilation, programmers develop a neural surrogate that replicates the behavior of the original program to deploy to end-users in place of the original program. With neural adaptation, programmers first develop a neural surrogate of a program then continue to train the neural surrogate on a different task. With neural surrogate optimization, programmers develop a neural surrogate of the original program, optimize input parameters of that neural surrogate using gradient descent, then plug the optimized input parameters back into the original program. Compared to standard programming approaches on these tasks, neural surrogates are more efficient and result in higher accuracy.

However, the approaches in the literature for developing neural surrogates are disparate. We identify the \emph{neural surrogate programming methodology} common to these approaches, consisting of the \emph{specification} of the problem, the \emph{design} of the neural network architecture, the \emph{training} of the neural network, and the \emph{deployment} considerations of the system.