From 0673f1ad120598a53a657003ab12c08a370d51f5 Mon Sep 17 00:00:00 2001 From: Leila Ghaffari Date: Thu, 11 Aug 2022 13:02:59 -0600 Subject: [PATCH 1/4] some intro --- content/getting_started/Examples.md | 17 +++++++++++++++++ content/getting_started/_index.md | 4 +++- 2 files changed, 20 insertions(+), 1 deletion(-) create mode 100644 content/getting_started/Examples.md diff --git a/content/getting_started/Examples.md b/content/getting_started/Examples.md new file mode 100644 index 0000000..1c0a773 --- /dev/null +++ b/content/getting_started/Examples.md @@ -0,0 +1,17 @@ +# Automatic Differentiation in Scientific Computing + +It goes without saying that `derivatives` are essential in formulating, hence simulating, physical phenomena. However, in real world problems, they can be complicated or error-prone to be deriven analytically, and numerical differentiation can introduce round-off errors in the discretization process and cancellation. These problems become more serious with higher derivatives beside being slow. Automatic differentiation, on the other hand, doesn't have any of these problems. + +Differentiation in general is all about chain rule. We know from Calculus that we can differentiate a given composition $h(g(f(x)))$ via + +$$ \operatorname d h = \frac{dh}{dg} \left(\frac{dg}{df} \left(\frac{df}{dx} \operatorname d x \right) \right) .$$ + +This method is called `forward-mode` and it is mostly used when there are one input and multiple outputs. While forward-mode differentiation is more efficient than other methods, the Scientific Community has shown more interest in cases where there are multiple inputs and a single output. This can be obtained via reverse-mode differentiation. Rearranging the parentheses in the expression above, we will have + +$$ \operatorname d h = \left( \left( \left( \frac{dh}{dg} \right) \frac{dg}{df} \right) \frac{df}{dx} \right) \operatorname d x $$ + +which we can compute with `reverse-mode` via + +$$ \underbrace{\bar x}_{\frac{dh}{dx}} = \underbrace{\bar g \frac{dg}{df}}_{\bar f} \frac{df}{dx} .$$ + +Enzyme is a primarily targeting reverse-mode differentiation, though it supports the forward-mode as well. In the following examples we show some common use cases in scientific computing with both modes. diff --git a/content/getting_started/_index.md b/content/getting_started/_index.md index 0c45c19..7464088 100644 --- a/content/getting_started/_index.md +++ b/content/getting_started/_index.md @@ -9,4 +9,6 @@ Please see the [Using Enzyme](/getting_started/UsingEnzyme) guide for informatio Please see the [Calling Convention](/getting_started/CallingConvention) guide for detailed information on Enzyme's calling convention and ABI, as well as how to specify how functions should be differentiated. -Please see the [CUDA Guide](/getting_started/CUDAGuide) guide for information on working with CUDA code. \ No newline at end of file +Please see the [CUDA Guide](/getting_started/CUDAGuide) guide for information on working with CUDA code. + +Please see [Examples](/getting_started/Examples) to see how Enzyme can be used in scientific applications. \ No newline at end of file From c13489a8f9dd97a6966bb2763c7994b1df82ca43 Mon Sep 17 00:00:00 2001 From: Leila Ghaffari Date: Mon, 15 Aug 2022 12:38:51 -0600 Subject: [PATCH 2/4] Add constant function --- content/getting_started/Examples.md | 30 ++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/content/getting_started/Examples.md b/content/getting_started/Examples.md index 1c0a773..cddff7b 100644 --- a/content/getting_started/Examples.md +++ b/content/getting_started/Examples.md @@ -1,6 +1,6 @@ # Automatic Differentiation in Scientific Computing -It goes without saying that `derivatives` are essential in formulating, hence simulating, physical phenomena. However, in real world problems, they can be complicated or error-prone to be deriven analytically, and numerical differentiation can introduce round-off errors in the discretization process and cancellation. These problems become more serious with higher derivatives beside being slow. Automatic differentiation, on the other hand, doesn't have any of these problems. +It goes without saying that `derivatives` are essential in formulating, hence simulating, physical phenomena. However, in real world problems, they can be complicated or error-prone to be derived analytically, and numerical differentiation can introduce round-off errors in the discretization process and cancellation. These problems become more serious with higher derivatives beside being slow. Automatic differentiation, on the other hand, doesn't have any of these problems. Differentiation in general is all about chain rule. We know from Calculus that we can differentiate a given composition $h(g(f(x)))$ via @@ -15,3 +15,31 @@ which we can compute with `reverse-mode` via $$ \underbrace{\bar x}_{\frac{dh}{dx}} = \underbrace{\bar g \frac{dg}{df}}_{\bar f} \frac{df}{dx} .$$ Enzyme is a primarily targeting reverse-mode differentiation, though it supports the forward-mode as well. In the following examples we show some common use cases in scientific computing with both modes. + +## Constant Function + +Let's start with the simplest case in calculus: constant functions/values. The following example demonstrates that the derivative of a constant function is, indeed, zero. + +```c +// constant_rev.c +// This function demonstrates the derivative of a constant function. + +#include +extern double __enzyme_autodiff(void *); +double Constant() { return 2.; } +double dConstant() { return __enzyme_autodiff((void *) Constant); } +int main() { + printf(" Constant = %f \n dConstant = %f \n", Constant(), dConstant()); +} +``` + +One way to compile this code is with the following arguments: +```bash +$ clang constant_rev.c -Xclang -load -Xclang ./Enzyme/ClangEnzyme-13.so -O2 -flegacy-pass-manager +``` + +Output: +```bash + Constant = 2.000000 + dConstant = 0.000000 +``` From e3ab27567fa00737336710c9b7e4ce29855bf18a Mon Sep 17 00:00:00 2001 From: Leila Ghaffari Date: Wed, 17 Aug 2022 14:27:02 -0600 Subject: [PATCH 3/4] Add a Examples tab --- content/getting_started/Examples.md | 7 +++++++ content/getting_started/Faq.md | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/content/getting_started/Examples.md b/content/getting_started/Examples.md index cddff7b..20e10e3 100644 --- a/content/getting_started/Examples.md +++ b/content/getting_started/Examples.md @@ -1,3 +1,10 @@ +--- +title: "Examples" +date: "2022-08-17" +draft: false +weight: 40 +--- + # Automatic Differentiation in Scientific Computing It goes without saying that `derivatives` are essential in formulating, hence simulating, physical phenomena. However, in real world problems, they can be complicated or error-prone to be derived analytically, and numerical differentiation can introduce round-off errors in the discretization process and cancellation. These problems become more serious with higher derivatives beside being slow. Automatic differentiation, on the other hand, doesn't have any of these problems. diff --git a/content/getting_started/Faq.md b/content/getting_started/Faq.md index 028e407..7d874d9 100644 --- a/content/getting_started/Faq.md +++ b/content/getting_started/Faq.md @@ -2,7 +2,7 @@ title: "FAQ" date: "2019-11-29" menu: "main" -weight: 40 +weight: 50 --- ## Enzyme builds successfully but won't run tests From d21d582971dc86ccc483ef84a06f980c38ee08b9 Mon Sep 17 00:00:00 2001 From: Leila Ghaffari Date: Thu, 18 Aug 2022 08:47:04 -0600 Subject: [PATCH 4/4] Examples: add scalar function --- content/getting_started/Examples.md | 41 +++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/content/getting_started/Examples.md b/content/getting_started/Examples.md index 20e10e3..8d80f8b 100644 --- a/content/getting_started/Examples.md +++ b/content/getting_started/Examples.md @@ -50,3 +50,44 @@ Output: Constant = 2.000000 dConstant = 0.000000 ``` + + +## Scalar Function + +The following example computes the volume of a cylinder, $v$, given $r$ (radius) and $h$ (height), and the constant value $\pi$. +Taking the derivative of $v$ with respect to $r$ and $h$, we obtain the lateral surface area and the cross-sectional surface area, respectively. Note that this is a multi-variable scalar function, for which we demonstrate how to treat constant/frozen variables (e.g. $\pi$) as well. + +```c +// scalar_rev.c +// This function demonstrates the derivative of a scalar function. +#include +extern double __enzyme_autodiff(void *, ...); +int enzyme_const; // This is a built-in Enzyme variable for frozen variables +void V_cylinder(double *v, double *r, double *h, const double pi) { + *v = pi * (*r) * (*r) * (*h); +} +void dV_cylinder(double *v, double *r, double *dr, double *h, double *dh, const double pi) { + double dv = 1; + __enzyme_autodiff((void *) V_cylinder, v, &dv, r, dr, h, dh, enzyme_const, pi); +} +int main() { + const double pi = 3.141593; + double r = 3, h = 2, volume_cylinder = 0, surface_lateral = 0, surface_cross_sectional = 0; + dV_cylinder(&volume_cylinder, &r, &surface_lateral, &h, &surface_cross_sectional, pi); + printf("Cylinder with r=%f and h=%f \n", r, h); + printf("----------------------------------------\n"); + printf("Volume = %f \n", volume_cylinder); + printf("Lateral Surface Area = %f \n", surface_lateral); + printf("Cross-sectional Surface Area = %f \n", surface_cross_sectional); + return 0; +} +``` + +Output: +```bash +Cylinder with r=3.000000 and h=2.000000 +---------------------------------------- +Volume = 56.548674 +Lateral Surface Area = 37.699116 +Cross-sectional Surface Area = 28.274337 +```