Neural Jump SDEs (Jump Diffusions) and Neural PDEs
June 5 2019 in Differential Equations, Julia, Mathematics, Stochastics | Tags: CUDA, differentiable programming, DifferentialEquations.jl, gpu, julia, stochastic differential equations | Author: Christopher Rackauckas
This is just an exploration of some new neural models I decided to jot down for safe keeping. DiffEqFlux.jl gives you the differentiable programming tools to allow you to use any DifferentialEquations.jl problem type (DEProblem) mixed with neural networks. We demonstrated this before, not just with neural ordinary differential equations, but also with things like neural stochastic differential equations and neural delay differential equations.
At the time we made DiffEqFlux, we were the “first to the gate” for many of these differential equations types and left it as an open question for people to find a use for these tools. And judging by the Arxiv papers that went out days after NeurIPS submissions were due, it looks like people now have justified some machine learning use cases for them. There were two separate papers on neural … READ MORE
Simulation and Control of Biological Stochasticity
June 5 2018 in Biology, Differential Equations, Mathematics, Stochastics | Tags: | Author: Christopher Rackauckas
Yesterday I defended my PhD thesis entitled “Simulation and Control of Biological Stochasticity”. Here’s the recording:
Solving Systems of Stochastic PDEs and using GPUs in Julia
December 11 2017 in CUDA, Differential Equations, Julia, Mathematics, Stochastics | Tags: | Author: Christopher Rackauckas
What I want to describe in this post is how to solve stochastic PDEs in Julia using GPU parallelism. I will go from start to finish, describing how to use the type-genericness of the DifferentialEquations.jl library in order to write a code that uses within-method GPU-parallelism on the system of PDEs. This is mostly a proof of concept: the most efficient integrators for this problem are not compatible with GPU parallelism yet, and the GPU parallelism isn’t fully efficient yet. However, I thought it would be nice to show an early progress report showing that it works and what needs to be fixed in Base Julia and various libraries for us to get the full efficiency.
Edit May 2019
As of DifferentialEquations.jl v6.4.0, this is no longer a proof of concept. The whole library, including implicit solvers with GMRES, etc., and … READ MORE
Video Introduction to DifferentialEquations.jl
July 7 2017 in Differential Equations, Julia, Mathematics, Programming, Stochastics | Tags: | Author: Christopher Rackauckas
Videos can be much easier to follow than text (though they usually have fewer details!). So, here’s a video introduction to DifferentialEquations.jl from JuliaCon. In this talk I walk through the major features of DifferentialEquations.jl by walking through the the tutorials in the documentation, highlighting usage details and explaining how to properly think about the code. I hope this helps make it easier to adopt DifferentialEquations.jl!
DifferentialEquations.jl 2.0: State of the Ecosystem
May 8 2017 in CUDA, Differential Equations, HPC, Julia, Mathematics, Stochastics, Uncategorized | Tags: | Author: Christopher Rackauckas
In this blog post I want to summarize what we have accomplished with DifferentialEquations’ 2.0 release and detail where we are going next. I want to put the design changes and development work into a larger context so that way everyone can better understand what has been achieved, and better understand how we are planning to tackle our next challenges.
If you find this project interesting and would like to support our work, please star our Github repository. Thanks!
Now let’s get started.
DifferentialEquations.jl 1.0: The Core
Before we start talking about 2.0, let’s understand first what 1.0 was all about. DifferentialEquations.jl 1.0 was about answering a single question: how can we put the wide array of differential equations into one simple and efficient interface. The result of this was the common interface explained in the first blog post. Essentially, we created … READ MORE
6 Months of DifferentialEquations.jl: Where We Are and Where We Are Going
December 15 2016 in Differential Equations, Julia, Mathematics, Programming, Stochastics, Uncategorized | Tags: DifferentialEquations.jl, julia | Author: Christopher Rackauckas
So around 6 months ago, DifferentialEquations.jl was first registered. It was at first made to be a library which can solve “some” types of differential equations, and that “some” didn’t even include ordinary differential equations. The focus was mostly fast algorithms for stochastic differential equations and partial differential equations.
Needless to say, Julia makes you too productive. Ambitions grew. By the first release announcement, much had already changed. Not only were there ordinary differential equation solvers, there were many. But the key difference was a change in focus. Instead of just looking to give a production-quality library of fast methods, a major goal of DifferentialEquations.jl became to unify the various existing packages of Julia to give one user-friendly interface.
Since that release announcement, we have made enormous progress. At this point, I believe we have both the most expansive and flexible … READ MORE
Introducing DifferentialEquations.jl
August 1 2016 in Differential Equations, FEM, Julia, Programming, Stochastics | Tags: DifferentialEquations.jl, julia | Author: Christopher Rackauckas
Edit: This post is very old. See this post for more up-to-date information.
Differential equations are ubiquitous throughout mathematics and the sciences. In fact, I myself have studied various forms of differential equations stemming from fields including biology, chemistry, economics, and climatology. What was interesting is that, although many different people are using differential equations for many different things, pretty much everyone wants the same thing: to quickly solve differential equations in their various forms, and make some pretty plots to describe what happened.
The goal of DifferentialEquations.jl is to do exactly that: to make it easy solve differential equations with the latest and greatest algorithms, and put out a pretty plot. The core idea behind DifferentialEquations.jl is that, while it is easy to describe a differential equation, they have such diverse behavior that experts have spent over a century compiling … READ MORE
Optimal Number of Workers for Parallel Julia
April 16 2016 in HPC, Julia, Programming, Stochastics | Tags: BLAS, hyperthreading, julia, parallel computing, workers | Author: Christopher Rackauckas
How many workers do you choose when running a parallel job in Julia? The answer is easy right? The number of physical cores. We always default to that number. For my Core i7 4770K, that means it’s 4, not 8 since that would include the hyperthreads. On my FX8350, there are 8 cores, but only 4 floating-point units (FPUs) which do the math, so in mathematical projects, I should use 4, right? I want to demonstrate that it’s not that simple.
Where the Intuition Comes From
Most of the time when doing scientific computing you are doing parallel programming without even knowing it. This is because a lot of vectorized operations are “implicitly paralleled”, meaning that they are multi-threaded behind the scenes to make everything faster. In other languages like Python, MATLAB, and R, this is also the case. Fire up MATLAB … READ MORE
Interfacing with a Xeon Phi via Julia
March 4 2016 in C, HPC, Julia, Programming, Stochastics, Xeon Phi | Tags: C, julia, MIC, OpenMP, parallel, Xeon Phi | Author: Christopher Rackauckas
(Disclaimer: This is not a full-Julia solution for using the Phi, and instead is a tutorial on how to link OpenMP/C code for the Xeon Phi to Julia. There may be a future update where some of these functions are specified in Julia, and Intel’s compilertools.jl looks like a viable solution, but for now it’s not possible.)
Intel’s Xeon Phi has a lot of appeal. It’s an instant cluster in your computer, right? It turns out it’s not quite that easy. For one, the installation process itself is quite tricky, and the device has stringent requirements for motherboard choices. Also, making out at over a taraflop is good, but not quite as high as NVIDIA’s GPU acceleration cards.
However, there are a few big reasons why I think our interest in the Xeon Phi should be renewed. For one, Intel … READ MORE
What Exactly is Brownian Motion and Why Does it Matter?
January 19 2016 in Stochastics | Tags: biology, brownian motion, nonstandard analysis, stochastic differential equations, Wiener process | Author: Christopher Rackauckas
When you talk about randomness, stochastics, and noise, everything always seems to be going back to “Brownian motion” and “white noise”. But what exactly are these things?
The problem is that if you ask someone who researches the subject, you most likely get many different definitions at the same time. The reason is because, although there are many equivalent definitions (and terms), every single one only seems to be a small part of the intuition of what this thing actually is. So I am going to state without proof equivalent formulations of $$B(t)$$, which is the same as $$B_{t}$$, which is the same as $$W_{t}$$… READ MORE