Skip to content

Commit c7f8361

Browse files
committed
Rewrite Data.IntMap to be faster and use less memory
Memory usage: The old implementation had 6n-5 words of overhead (i.e. discounting storage of the keys and the pointers to values). The new implementation has only 3n-1 words of overhead. Large runtime regressions: - fromAscList and fromDistinctAscList are currently just aliases for fromList - I have no clue about foldlWithKey' - keysSet, fromSet, restrictKeys, and withoutKeys are currently implemented naively and are probably much slower Benchmarks after: Benchmark intmap-benchmarks: RUNNING... benchmarking lookup time 351.2 μs (348.0 μs .. 353.4 μs) 0.999 R² (0.999 R² .. 1.000 R²) mean 348.4 μs (345.8 μs .. 350.5 μs) std dev 7.589 μs (5.950 μs .. 8.951 μs) variance introduced by outliers: 14% (moderately inflated) benchmarking insert time 710.6 μs (708.1 μs .. 713.5 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 709.2 μs (707.8 μs .. 711.0 μs) std dev 5.133 μs (4.064 μs .. 6.746 μs) benchmarking insertWith empty time 784.2 μs (782.1 μs .. 786.3 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 783.8 μs (781.9 μs .. 786.2 μs) std dev 7.298 μs (5.164 μs .. 10.90 μs) benchmarking insertWith update time 1.815 ms (1.809 ms .. 1.823 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.815 ms (1.810 ms .. 1.819 ms) std dev 16.52 μs (12.94 μs .. 22.60 μs) benchmarking insertWith' empty time 794.3 μs (792.9 μs .. 795.7 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 794.5 μs (792.8 μs .. 796.1 μs) std dev 6.120 μs (4.768 μs .. 8.325 μs) benchmarking insertWith' update time 1.491 ms (1.486 ms .. 1.496 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.496 ms (1.491 ms .. 1.503 ms) std dev 18.86 μs (14.67 μs .. 24.30 μs) benchmarking insertWithKey empty time 790.0 μs (786.0 μs .. 793.7 μs) 1.000 R² (0.999 R² .. 1.000 R²) mean 794.1 μs (790.1 μs .. 807.6 μs) std dev 22.04 μs (5.555 μs .. 45.51 μs) variance introduced by outliers: 18% (moderately inflated) benchmarking insertWithKey update time 2.034 ms (2.029 ms .. 2.042 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 2.033 ms (2.028 ms .. 2.039 ms) std dev 17.85 μs (14.47 μs .. 22.44 μs) benchmarking insertWithKey' empty time 797.9 μs (796.6 μs .. 798.9 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 797.3 μs (795.8 μs .. 798.9 μs) std dev 5.257 μs (4.250 μs .. 6.617 μs) benchmarking insertWithKey' update time 1.504 ms (1.499 ms .. 1.510 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.504 ms (1.500 ms .. 1.511 ms) std dev 18.15 μs (12.36 μs .. 29.84 μs) benchmarking insertLookupWithKey empty time 917.9 μs (914.9 μs .. 920.7 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 915.4 μs (912.2 μs .. 918.1 μs) std dev 10.09 μs (8.160 μs .. 12.89 μs) benchmarking insertLookupWithKey update time 2.332 ms (2.326 ms .. 2.338 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 2.334 ms (2.329 ms .. 2.341 ms) std dev 20.29 μs (16.52 μs .. 24.81 μs) benchmarking map time 149.6 μs (149.1 μs .. 150.0 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 151.7 μs (151.0 μs .. 153.6 μs) std dev 3.726 μs (1.409 μs .. 7.458 μs) variance introduced by outliers: 19% (moderately inflated) benchmarking mapWithKey time 162.1 μs (161.9 μs .. 162.3 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 162.1 μs (162.0 μs .. 162.4 μs) std dev 623.3 ns (379.3 ns .. 1.039 μs) benchmarking foldlWithKey time 1.188 ms (1.184 ms .. 1.193 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.190 ms (1.187 ms .. 1.194 ms) std dev 10.63 μs (8.096 μs .. 14.74 μs) benchmarking foldlWithKey' time 93.76 μs (93.72 μs .. 93.88 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 93.80 μs (93.74 μs .. 93.98 μs) std dev 371.8 ns (30.91 ns .. 714.5 ns) benchmarking foldrWithKey time 22.71 ns (22.62 ns .. 22.80 ns) 0.998 R² (0.996 R² .. 1.000 R²) mean 22.65 ns (22.13 ns .. 23.43 ns) std dev 2.077 ns (1.152 ns .. 3.451 ns) variance introduced by outliers: 90% (severely inflated) benchmarking delete time 392.1 μs (384.0 μs .. 408.8 μs) 0.994 R² (0.985 R² .. 1.000 R²) mean 386.7 μs (384.2 μs .. 395.9 μs) std dev 14.89 μs (710.6 ns .. 31.54 μs) variance introduced by outliers: 32% (moderately inflated) benchmarking update time 1.277 ms (1.275 ms .. 1.280 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.277 ms (1.274 ms .. 1.279 ms) std dev 8.210 μs (6.086 μs .. 11.37 μs) benchmarking updateLookupWithKey time 2.538 ms (2.529 ms .. 2.547 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 2.534 ms (2.527 ms .. 2.540 ms) std dev 22.10 μs (18.13 μs .. 28.85 μs) benchmarking alter time 1.555 ms (1.550 ms .. 1.559 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.554 ms (1.550 ms .. 1.558 ms) std dev 13.60 μs (11.01 μs .. 16.65 μs) benchmarking mapMaybe time 184.7 μs (184.1 μs .. 185.3 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 185.1 μs (184.5 μs .. 185.9 μs) std dev 2.239 μs (1.418 μs .. 3.591 μs) benchmarking mapMaybeWithKey time 186.9 μs (184.6 μs .. 191.9 μs) 0.992 R² (0.977 R² .. 1.000 R²) mean 187.6 μs (185.1 μs .. 197.7 μs) std dev 14.17 μs (2.403 μs .. 31.55 μs) variance introduced by outliers: 69% (severely inflated) benchmarking fromList time 717.2 μs (713.4 μs .. 719.9 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 713.2 μs (710.5 μs .. 716.1 μs) std dev 9.082 μs (7.906 μs .. 10.82 μs) benchmarking fromAscList time 718.4 μs (713.7 μs .. 722.3 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 712.8 μs (710.4 μs .. 715.4 μs) std dev 8.612 μs (7.509 μs .. 10.04 μs) benchmarking fromDistinctAscList time 717.4 μs (713.4 μs .. 720.7 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 712.6 μs (710.7 μs .. 714.8 μs) std dev 6.553 μs (5.526 μs .. 7.963 μs) Benchmark intmap-benchmarks: FINISH Benchmarks before: Benchmark intmap-benchmarks: RUNNING... benchmarking lookup time 462.8 μs (457.1 μs .. 466.8 μs) 0.999 R² (0.998 R² .. 1.000 R²) mean 461.9 μs (457.6 μs .. 464.5 μs) std dev 10.83 μs (7.533 μs .. 14.53 μs) variance introduced by outliers: 15% (moderately inflated) benchmarking insert time 908.2 μs (906.9 μs .. 909.8 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 908.7 μs (907.0 μs .. 910.9 μs) std dev 6.776 μs (5.050 μs .. 9.429 μs) benchmarking insertWith empty time 904.2 μs (903.1 μs .. 905.6 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 903.8 μs (902.0 μs .. 905.8 μs) std dev 5.991 μs (4.748 μs .. 7.628 μs) benchmarking insertWith update time 1.962 ms (1.957 ms .. 1.968 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.965 ms (1.959 ms .. 1.971 ms) std dev 18.44 μs (13.84 μs .. 24.77 μs) benchmarking insertWith' empty time 918.0 μs (916.0 μs .. 920.9 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 918.9 μs (915.3 μs .. 923.0 μs) std dev 13.87 μs (10.57 μs .. 18.18 μs) benchmarking insertWith' update time 1.682 ms (1.672 ms .. 1.706 ms) 0.998 R² (0.995 R² .. 1.000 R²) mean 1.686 ms (1.678 ms .. 1.709 ms) std dev 44.73 μs (18.01 μs .. 85.74 μs) variance introduced by outliers: 14% (moderately inflated) benchmarking insertWithKey empty time 906.4 μs (905.0 μs .. 907.8 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 905.8 μs (904.3 μs .. 907.2 μs) std dev 5.118 μs (4.080 μs .. 6.452 μs) benchmarking insertWithKey update time 2.039 ms (2.019 ms .. 2.054 ms) 1.000 R² (0.999 R² .. 1.000 R²) mean 2.021 ms (2.014 ms .. 2.029 ms) std dev 25.52 μs (20.98 μs .. 31.95 μs) benchmarking insertWithKey' empty time 912.6 μs (910.4 μs .. 916.3 μs) 1.000 R² (0.999 R² .. 1.000 R²) mean 925.0 μs (919.7 μs .. 930.8 μs) std dev 18.31 μs (15.26 μs .. 21.70 μs) benchmarking insertWithKey' update time 1.690 ms (1.687 ms .. 1.695 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.693 ms (1.689 ms .. 1.699 ms) std dev 17.40 μs (13.96 μs .. 23.11 μs) benchmarking insertLookupWithKey empty time 1.772 ms (1.766 ms .. 1.778 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.771 ms (1.768 ms .. 1.774 ms) std dev 10.39 μs (9.049 μs .. 12.27 μs) benchmarking insertLookupWithKey update time 4.051 ms (4.019 ms .. 4.092 ms) 1.000 R² (0.999 R² .. 1.000 R²) mean 4.031 ms (4.018 ms .. 4.048 ms) std dev 50.84 μs (38.55 μs .. 67.42 μs) benchmarking map time 172.7 μs (171.4 μs .. 173.7 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 171.3 μs (171.0 μs .. 171.9 μs) std dev 1.377 μs (870.7 ns .. 1.961 μs) benchmarking mapWithKey time 227.8 μs (226.9 μs .. 228.8 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 227.1 μs (226.6 μs .. 227.7 μs) std dev 1.799 μs (1.315 μs .. 2.385 μs) benchmarking foldlWithKey time 1.554 ms (1.513 ms .. 1.618 ms) 0.995 R² (0.991 R² .. 1.000 R²) mean 1.546 ms (1.538 ms .. 1.566 ms) std dev 40.51 μs (21.11 μs .. 81.14 μs) variance introduced by outliers: 14% (moderately inflated) benchmarking foldlWithKey' time 46.11 μs (45.90 μs .. 46.25 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 45.64 μs (45.52 μs .. 45.78 μs) std dev 421.3 ns (342.6 ns .. 499.9 ns) benchmarking foldrWithKey time 105.9 ns (105.7 ns .. 106.3 ns) 1.000 R² (1.000 R² .. 1.000 R²) mean 105.9 ns (105.7 ns .. 106.0 ns) std dev 461.7 ps (320.4 ps .. 689.5 ps) benchmarking delete time 357.5 μs (356.7 μs .. 359.0 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 357.6 μs (357.3 μs .. 358.5 μs) std dev 1.734 μs (1.000 μs .. 3.244 μs) benchmarking update time 1.542 ms (1.539 ms .. 1.548 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.541 ms (1.538 ms .. 1.544 ms) std dev 10.09 μs (7.858 μs .. 13.96 μs) benchmarking updateLookupWithKey time 2.444 ms (2.439 ms .. 2.448 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 2.457 ms (2.451 ms .. 2.467 ms) std dev 24.38 μs (17.83 μs .. 32.20 μs) benchmarking alter time 1.536 ms (1.532 ms .. 1.540 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.538 ms (1.535 ms .. 1.541 ms) std dev 10.88 μs (8.249 μs .. 16.94 μs) benchmarking mapMaybe time 223.8 μs (223.6 μs .. 224.0 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 224.4 μs (223.5 μs .. 226.4 μs) std dev 4.150 μs (616.6 ns .. 7.538 μs) variance introduced by outliers: 11% (moderately inflated) benchmarking mapMaybeWithKey time 223.3 μs (223.0 μs .. 223.6 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 223.3 μs (223.1 μs .. 223.6 μs) std dev 749.0 ns (456.5 ns .. 1.329 μs) benchmarking fromList time 848.4 μs (847.1 μs .. 849.4 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 847.0 μs (845.7 μs .. 848.5 μs) std dev 4.540 μs (3.715 μs .. 6.255 μs) benchmarking fromAscList time 604.2 μs (602.4 μs .. 605.6 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 602.6 μs (601.4 μs .. 604.0 μs) std dev 4.334 μs (3.383 μs .. 5.573 μs) benchmarking fromDistinctAscList time 226.4 μs (225.7 μs .. 227.7 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 226.0 μs (225.7 μs .. 226.4 μs) std dev 988.2 ns (534.4 ns .. 1.965 μs) Benchmark intmap-benchmarks: FINISH
1 parent 0d83f53 commit c7f8361

File tree

8 files changed

+4799
-2817
lines changed

8 files changed

+4799
-2817
lines changed

Data/IntMap.hs

+6-20
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
-- Module : Data.IntMap
1111
-- Copyright : (c) Daan Leijen 2002
1212
-- (c) Andriy Palamarchuk 2008
13+
-- (c) Jonathan S. 2016
1314
-- License : BSD-style
1415
-- Maintainer : [email protected]
1516
-- Stability : provisional
@@ -31,20 +32,6 @@
3132
-- > import Data.IntMap (IntMap)
3233
-- > import qualified Data.IntMap as IntMap
3334
--
34-
-- The implementation is based on /big-endian patricia trees/. This data
35-
-- structure performs especially well on binary operations like 'union'
36-
-- and 'intersection'. However, my benchmarks show that it is also
37-
-- (much) faster on insertions and deletions when compared to a generic
38-
-- size-balanced map implementation (see "Data.Map").
39-
--
40-
-- * Chris Okasaki and Andy Gill, \"/Fast Mergeable Integer Maps/\",
41-
-- Workshop on ML, September 1998, pages 77-86,
42-
-- <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.37.5452>
43-
--
44-
-- * D.R. Morrison, \"/PATRICIA -- Practical Algorithm To Retrieve
45-
-- Information Coded In Alphanumeric/\", Journal of the ACM, 15(4),
46-
-- October 1968, pages 514-534.
47-
--
4835
-- Operation comments contain the operation time complexity in
4936
-- the Big-O notation <http://en.wikipedia.org/wiki/Big_O_notation>.
5037
-- Many operations have a worst-case complexity of /O(min(n,W))/.
@@ -61,37 +48,36 @@ module Data.IntMap
6148
, foldWithKey
6249
) where
6350

64-
import Prelude () -- hide foldr
51+
import Prelude hiding (foldr)
6552
import qualified Data.IntMap.Strict as Strict
6653
import Data.IntMap.Lazy
6754

6855
-- | /O(log n)/. Same as 'insertWith', but the result of the combining function
6956
-- is evaluated to WHNF before inserted to the map.
70-
7157
{-# DEPRECATED insertWith' "As of version 0.5, replaced by 'Data.IntMap.Strict.insertWith'." #-}
58+
{-# INLINE insertWith' #-}
7259
insertWith' :: (a -> a -> a) -> Key -> a -> IntMap a -> IntMap a
7360
insertWith' = Strict.insertWith
7461

7562
-- | /O(log n)/. Same as 'insertWithKey', but the result of the combining
7663
-- function is evaluated to WHNF before inserted to the map.
77-
7864
{-# DEPRECATED insertWithKey' "As of version 0.5, replaced by 'Data.IntMap.Strict.insertWithKey'." #-}
65+
{-# INLINE insertWithKey' #-}
7966
insertWithKey' :: (Key -> a -> a -> a) -> Key -> a -> IntMap a -> IntMap a
8067
insertWithKey' = Strict.insertWithKey
8168

8269
-- | /O(n)/. Fold the values in the map using the given
8370
-- right-associative binary operator. This function is an equivalent
8471
-- of 'foldr' and is present for compatibility only.
8572
{-# DEPRECATED fold "As of version 0.5, replaced by 'foldr'." #-}
73+
{-# INLINE fold #-}
8674
fold :: (a -> b -> b) -> b -> IntMap a -> b
8775
fold = foldr
88-
{-# INLINE fold #-}
8976

9077
-- | /O(n)/. Fold the keys and values in the map using the given
9178
-- right-associative binary operator. This function is an equivalent
9279
-- of 'foldrWithKey' and is present for compatibility only.
93-
9480
{-# DEPRECATED foldWithKey "As of version 0.5, replaced by 'foldrWithKey'." #-}
81+
{-# INLINE foldWithKey #-}
9582
foldWithKey :: (Key -> a -> b -> b) -> b -> IntMap a -> b
9683
foldWithKey = foldrWithKey
97-
{-# INLINE foldWithKey #-}

0 commit comments

Comments
 (0)