Skip to content

Commit 772e124

Browse files
committed
Fixed formatting in HowToProfileLedger.md
1 parent f0d090c commit 772e124

File tree

1 file changed

+75
-73
lines changed

1 file changed

+75
-73
lines changed

HowToProfileLedger.md

+75-73
Original file line numberDiff line numberDiff line change
@@ -1,67 +1,67 @@
1-
How To Profile Ledger
2-
last edited
3-
May 7, 2022
4-
Tim Sheard
1+
# How To Profile Ledger
2+
*Tim Sheard*
53

6-
Motivation
7-
Profiling the ledger is an intricate dance between nix, cabal, and ghc. This document
4+
## Motivation
5+
6+
Profiling the ledger is an intricate dance between `nix`, `cabal`, and `ghc`. This document
87
describes how I set all this up to profile some of the property tests in the
9-
cardano-ledger-test module. I hope this is useful to others who want to profile
10-
other parts of the Ledger code.
8+
`cardano-ledger-test` module. I hope this is useful to others who want to profile
9+
other parts of the ledger codebase.
10+
11+
## Background
1112

12-
Background
13-
Profiling is an important tool to analyze performance (time and space) in Haskel code.
14-
I recommend reading through the GHC users guide. Here is a link to the appropriates section
15-
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/profiling.html
16-
This is great background, but the gude says almost nothing about how to make it all work
17-
in a repository that uses nix and cabal. Hence this document
13+
Profiling is an important tool to analyze performance (time and space) in Haskell code.
14+
I recommend reading through the [GHC users guide](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/profiling.html).
1815

16+
This is great background, but the guide says almost nothing about how to make it all work
17+
in a repository that uses `nix` and `cabal`. Hence this document
18+
19+
## Summary
1920

20-
Summary
2121
We list here a high level description of the steps needed. In further sections we go into
2222
greater detail about each step
2323

2424
1. Decide what you want to profile, and arrange the code so this is possible
25-
2. Choreograph the dance between nix, cabal, and ghc. This has two parts
26-
a. Adding stuff to files like cabal.project, cabal.project.local
27-
b. Passing the right flags to cabal, and ghc, running the right `nix develop` shell,
25+
2. Choreograph the dance between `nix`, `cabal`, and `ghc`. This has two parts
26+
1. Adding stuff to files like `cabal.project`, `cabal.project.local`
27+
2. Passing the right flags to `cabal`, and `ghc`, running the right `nix develop` shell,
2828
3. Recompiling everything so it can be profiled. This takes a very long time (greater than
2929
30 minutes when I did it)
30-
4. Start up the profiling. I used "cabal test" with just the right command line arguments.
31-
5. Inspect the produced .prof file, and decide what to do
30+
4. Start up the profiling. I used `cabal test` with just the right command line arguments.
31+
5. Inspect the produced `.prof` file, and decide what to do
3232
6. Repeat steps 3-5, until satisfied.
3333
7. Undo all the changes (from steps 1 and 2) made just for profiling.
3434

35-
Deciding what to profile
36-
Usually one wants to profile code that appears to be using too many resoures (space or time).
37-
Because of the difficulty in getting nix, cabal, and ghc to work together, I have found that
38-
co-opting an existing Test file is the way to go. Here are the reasons why this is a good idea
35+
## Deciding what to profile
36+
Usually one wants to profile code that appears to be using too many resources (space or time).
37+
Because of the difficulty in getting `nix`, `cabal`, and `ghc` to work together, I have found that
38+
co-opting an existing `Test.hs` file is the way to go. Here are the reasons why this is a good idea
3939

40-
1) Everything is already set up to compile and run the test file by simply typing: cabal test
40+
1) Everything is already set up to compile and run the test file by simply typing: `cabal test`
4141
in the root directory of the module that contains the test.
42-
2) Is is usually quite easy to rename the 'main' variable in the test file to something else
43-
and add your own 'main', that contains the code you want to profile
44-
3) Using a quick check property test, allows for your code to consist of mutltiple "tests" each
42+
2) Is is usually quite easy to rename the `main` variable in the test file to something else
43+
and add your own `main`, that contains the code you want to profile
44+
3) Using a quick check property test, allows for your code to consist of multiple "tests" each
4545
with a different random input. This means your profiling results are less likely to be biased
4646
by a bad choice of input.
4747

48-
Here is how I did it. I edited the file cardano-ledger/libs/cardano-ledger-test/test/Test.hs
48+
Here is how I did it. I edited the file `cardano-ledger/libs/cardano-ledger-test/test/Test.hs`
4949
Here is a synopsis of what it originally looked like, and what it looked like after I edited it.
5050

5151
BEFORE
5252
------------------------
53-
...
54-
53+
```haskell
5554
-- main entry point
5655
main :: IO ()
5756
main = do
5857
hSetEncoding stdout utf8
5958
defaultMain tests
59+
```
6060
------------------------
6161

6262
AFTER
6363
-----------------------
64-
...
64+
```haskell
6565
import Test.Cardano.Ledger.Generic.Properties (adaIsPreservedBabbage)
6666

6767
mainSave :: IO ()
@@ -71,87 +71,89 @@ mainSave = do
7171

7272
main :: IO ()
7373
main = defaultMain adaIsPreservedBabbage
74+
```
7475
-----------------------
7576

76-
The variable 'adaIsPreservedBabbage' is a property test of type 'TestTree' that takes
77+
The variable `adaIsPreservedBabbage` is a property test of type `TestTree` that takes
7778
about 1 minute to run. Similar tests took about 10-20 seconds, so I was interested why
78-
it took so long. So now I have a 'main' that runs just this one test. I could see that I
79-
have set things up properly by changing directory to cardano-ledger/libs/cardano-ledger-test
80-
The root directory of the module. This is the directory that contains the cardano-ledger-test.cabal file.
81-
So I can simply type: cabal test, and the test runs. Now all I need to do is get it to be profiled.
79+
it took so long. So now I have a `main` that runs just this one test. I could see that I
80+
have set things up properly by changing directory to `cardano-ledger/libs/cardano-ledger-test`,
81+
the root directory of the module. This is the directory that contains the `cardano-ledger-test.cabal` file.
82+
So I can simply type: `cabal test`, and the test runs. Now all I need to do is get it to be profiled.
8283

83-
Choreographing the dance.
84+
## Choreographing the dance
8485

85-
We need to tell nix, cabal, and ghc that we want to profile. We do this by changing a few of the
86-
files that build the system, and by passing the right flags to nix, cabal, and GHC.
86+
We need to tell `nix`, `cabal`, and `ghc` that we want to profile. We do this by changing a few of the
87+
files that build the system, and by passing the right flags to each of the
88+
programs.
8789

8890

89-
First we must add (or change if you already have it) cabal.project.local In the root of the
90-
ledger repository. Put this in cabal.project.local
91-
---------------------
91+
First we must add (or change if you already have it) `cabal.project.local` In the root of the
92+
ledger repository. Put this in `cabal.project.local`
93+
```cabal
9294
ignore-project: False
9395
profiling: True
9496
profiling-detail: all-functions
95-
---------------------
97+
```
9698

97-
The final step is to pass the right flags to cabal, ghc, and running the right `nix develop` shell. Here is a summary.
99+
The final step is to pass the right flags to `cabal`, `ghc`, and running the right `nix develop` shell. Here is a summary.
98100

99-
1) to start nix, we must use
101+
1) to start `nix`, we must use
100102
`nix develop .#profiling`
101-
2) to build with cabal, we must use
103+
2) to build with `cabal`, we must use
102104
`cabal build --enable-profiling`
103-
3) to run the test, we must pass extra flags to ghc, so we must use
105+
3) to run the test, we must pass extra flags to `ghc`, so we must use
104106
`cabal test --test-options="+RTS -i60 -p"`
105107

106-
How to build the system for profiling
108+
## How to build the system for profiling
107109

108-
Be sure you have set up the files cabal.project.local and nix/haskell.nix as described above.
109-
Exit the nix shell, if you are running it. Now change directories to the root of the Ledger repository.
110+
Be sure you have set up the files `cabal.project.local` and `nix/haskell.nix` as described above.
111+
Exit the `nix` shell, if you are running it. Now change directories to the root of the `cardano-ledger` repository.
110112

111-
Now to start nix type
112-
```
113+
Now to start `nix` type
114+
```bash
113115
nix develop .#profiling
114116
```
115-
(or, eg. `nix develop .#ghc8107.profiling` for alternative compiler)
117+
(or, e.g. `nix develop .#ghc8107.profiling` for alternative compiler)
116118

117119
When the `nix develop` shell completes (this can take a long time, since it must make sure
118-
every file in the Ledger is compiled with profiling enabled). This might take a
120+
every file in the ledger is compiled with profiling enabled). This might take a
119121
while. Be patient. Take the dogs for a walk.
120122

121123
If `nix develop .#profiling` fails to give you a nix shell this may be related to a problem
122-
with 'plutus-core' that uses template haskell, and happens to trigger a known bug in ghc.
123-
https://gitlab.haskell.org/ghc/ghc/-/issues/18320 There are also a few slack threads about this. For example
124-
https://input-output-rnd.slack.com/archives/C21UF2WVC/p1624555583258300
125-
The workaround for this is quite convoluted, but if you fail to get a nix-shell working there are two things
124+
with `plutus-core` that uses template Haskell, and happens to trigger a
125+
[known bug](https://gitlab.haskell.org/ghc/ghc/-/issues/18320) in `ghc`.
126+
127+
The workaround for this is quite convoluted, but if you fail to get a `nix-shell` working there are two things
126128
you may need to do:
127129

128130
1) add the following to `cabal.project.local`:
129-
```
131+
```cabal
130132
package plutus-core
131133
ghc-options: -fexternal-interpreter
132134
```
133135
2) uncomment the following in `flake.nix`:
134-
```
135-
# packages.plutus-core.components.library.ghcOptions = [ "-fexternal-interpreter" ];
136+
```nix
137+
packages.plutus-core.components.library.ghcOptions = [ "-fexternal-interpreter" ];
136138
```
137139

138-
Now change directories to the root directory of the modlue that contains your
139-
modified Test file, and type
140-
```
140+
Now change directories to the root directory of the module that contains your
141+
modified `Test.hs` file, and type
142+
```bash
141143
cabal build --enable-profiling
142144
```
143145
This might also take a while. Take the dogs for second walk. When this completes
144-
you are ready to start profiing!
146+
you are ready to start profiling!
145147

146-
How to run a profile.
148+
## How to run a profile
147149

148-
In the same directory where you did (cabal build --enable-profiling) type
149-
```
150+
In the same directory where you did (`cabal build --enable-profiling`) type
151+
```bash
150152
cabal test --test-options="+RTS -i60 -p"
151153
```
152154
This should take slightly longer than running the test without profiling.
153155
When it is done, there will be a file in this same directory with extension .prof
154-
When I did it, the file was called cardano-ledger-test.prof . It is a big file
156+
When I did it, the file was called `cardano-ledger-test.prof`. It is a big file
155157
Here are the first few lines.
156158
```
157159
----------------------------------------------------------------------------------------------------------
@@ -183,8 +185,8 @@ toLazyByteString Codec.CBOR.Write src/Codec/CBOR/W
183185
---------------------------------------------------------------------------------------------------------------------
184186
```
185187

186-
The problem with my test, was that evalScripts was inadvertantly showing a large data structure
187-
using tellEvent. This was added when debugging and never removed. After fixing this, we had much better results.
188+
The problem with my test, was that `evalScripts` was inadvertently showing a large data structure
189+
using `tellEvent`. This was added when debugging and never removed. After fixing this, we had much better results.
188190
I hope you experience is just as rewarding.
189191

190-
Don't forget to revert cabal.project.local, nix/haskell.nix and your Test file to their original state.
192+
Don't forget to revert `cabal.project.local`, `nix/haskell.nix` and your `Test.hs` file to their original state.

0 commit comments

Comments
 (0)