Abstract: Scientific machine learning (SciML) methods allow for the automatic discovery of mechanistic models by infusing neural network training into the simulation process. In this talk we will start by showcasing some of the ways that SciML is being used, from discovery of extrapolatory epidemic models to nonlinear mixed effects models in pharmacology. From there, we will discuss some of the increasingly advanced computational techniques behind the training process, focusing on the numerical issues involved in handling differentiation of highly stiff and chaotic systems. The viewers will leave with an understanding of how compiler techniques are being infused into the simulation stack to increasingly automate the process of developing mechanistic models

Bio: Dr. Chris Rackauckas is the Director of Scientific Research at Pumas-AI, the Director of Modeling … READ MORE

You may have seen some hilariously long stacktraces when using ForwardDiff. In the latest releases of OrdinaryDiffEq.jl we have fixed this, and the fix is rather safe. I want to take a second to describe some of the technical details so that others can copy this technique.

The reason for this is the tag parameter. The Dual number type is given by Dual{T,V,N} where V is an element type (usually Float64), N is a chunksize (some integer), and T is the tag. What the tag does is prevent perturbation confusion by erroring if two incompatible dual numbers try to interact. The key requirement for it to prevent perturbation confusion is for the type to be unique in the context of the user. For example, if the user is differentiating f, and then differentiating x->derivative(f,x), you want the tag to be … READ MORE

I recently saw a paper that made the following statement:

“Innes et al. [22] trained neural SDEs by backpropagating through the operations of the solver, however their training objective simply matched the first two moments of the training data, implying that it could not consistently estimate diffusion functions.”

However, given that the statement “could not consistently estimate diffusion functions” had no reference to it and no proof in the appendix, I was interested to figure out the mathematical foundation behind the claim. Furthermore, I know from the DiffEqFlux documentation example that there was at least one case where second order method of moments seems to estimate the diffusion function. So a question arose, when does the mean and variance define an SDE?

Of course, in 2021 a Twitter thread captures the full discussion. But I want to take a step back … READ MORE

Modeling practice seems to be partitioned into scientific models defined by mechanistic differential equations and machine learning models defined by parameterizations of neural networks. While the ability for interpretable mechanistic models to extrapolate from little information is seemingly at odds with the big data “model-free” approach of neural networks, the next step in scientific progress is to utilize these methodologies together in order to emphasize their strengths while mitigating weaknesses. In this talk we will describe four separate ways that we are merging differential equations and deep learning through the power of the DifferentialEquations.jl and Flux.jl libraries. Data-driven hypothesis generation of model structure, automated real-time control of dynamical systems, accelerated of PDE solving, and memory-efficient deep learning workflows will all shown to be derived from this common computational structure … READ MORE