@@ -25,6 +25,7 @@ <h1 id="status">Status</h1>
25
25
< li > Draft #1 published: 2019/6/24</ li >
26
26
< li > Draft #2 published: 2019/7/25</ li >
27
27
< li > Draft #3 published: 2019/10/4</ li >
28
+
28
29
</ ul >
29
30
30
31
< h1 id ="abstract "> Abstract</ h1 >
@@ -39,13 +40,14 @@ <h1 id="abstract">Abstract</h1>
39
40
< h1 id ="rationale "> Rationale</ h1 >
40
41
41
42
< p > Some of the most common operations used in the Scheme language are
42
- those transforming lists: < code > map</ code > , < code > filter</ code > ,
43
- < code > take</ code > and so on. They work well, are well understood,
44
- and are used daily by most Scheme programmers. They are however not
45
- general because they only work on lists, and they do not compose
46
- very well since combining < code > N</ code > of them builds < code > (- N
43
+ those transforming lists: < code > map</ code > , < code > filter</ code > ,
44
+ < code > take</ code > and so on. They work well, are well understood,
45
+ and are used daily by most Scheme programmers. They are however not
46
+ general because they only work on lists, and they do not compose
47
+ very well since combining < code > N</ code > of them builds < code > (- N
47
48
1)</ code > intermediate lists.</ p >
48
49
50
+
49
51
< p > Transducers are oblivious to what kind of process they are used in,
50
52
and are composable without building intermediate collections. This
51
53
means we can create a transducer that squares all even numbers:
@@ -72,7 +74,7 @@ <h2 id="dependencies">Dependencies</h2>
72
74
73
75
< ul >
74
76
< li > SRFI 9, < code > define-record-type</ code > (included in R< sup > 7</ sup > RS
75
- small)</ li >
77
+ small)</ li >
76
78
< li > SRFI-69 (hash-tables)</ li >
77
79
< li > Proper compose procedure (included if it is not available)</ li >
78
80
< li > A < code > vector->list</ code > that behaves like in SRFI 43
@@ -81,7 +83,7 @@ <h2 id="dependencies">Dependencies</h2>
81
83
< h2 id ="portability "> Portability</ h2 >
82
84
83
85
< p > The sample implementation is easily portable to any
84
- r5rs/r6rs/R < sup > 7</ sup > RS -compatible Scheme. The non-standard things are:</ p >
86
+ r < sup > 5 </ sup > rs/r < sup > 6 </ sup > rs/r < sup > 7</ sup > rs -compatible Scheme. The non-standard things are:</ p >
85
87
86
88
< ul >
87
89
< li > a < code > vector->list</ code > that takes start and end
@@ -127,25 +129,26 @@ <h2 id="concept-transducers">Concept: Transducers</h2>
127
129
< em > result-so-far</ em > and the maybe-transformed
128
130
< em > input</ em > .</ li > </ ul >
129
131
130
- < p > A simple example is as following: < code > (list-transduce (tfilter
132
+
133
+ < p > A simple example is as following: < code > (list-transduce (tfilter
131
134
odd?) + '(1 2 3 4 5))</ code > . This first returns a transducer
132
- filtering all odd elements, then it runs < code > +</ code > without
133
- arguments to retrieve its identity. It then starts the transduction
134
- by passing + to the transducer returned by < code > (tfilter
135
+ filtering all odd elements, then it runs < code > +</ code > without
136
+ arguments to retrieve its identity. It then starts the transduction
137
+ by passing + to the transducer returned by < code > (tfilter
135
138
odd?)</ code > which returns a reducing function. It works not unlike
136
- < code > reduce</ code > from SRFI 1, but also checks whether one of the
137
- intermediate transducers returns a "reduced" value
138
- (implemented as a SRFI 9 record), which means the reduction finished
139
- early.</ p >
139
+ < code > reduce</ code > from SRFI 1, but also checks whether one of the
140
+ intermediate transducers returns a "reduced" value
141
+ (implemented as a SRFI 9 record), which means the reduction finished
142
+ early.</ p >
140
143
141
144
< p > Because transducers compose and the final reduction is only
142
- executed in the last step, composed transducers will not build any
143
- intermediate result or collections. Although the normal way of
144
- thinking about application of composed functions is right to left,
145
- due to how the transduction is built it is applied left to
146
- right. < code > (compose (tfilter odd?) (tmap sqrt))</ code > will
147
- create a transducer that first filters out any odd values and then
148
- computes the square root of the rest.</ p >
145
+ executed in the last step, composed transducers will not build any
146
+ intermediate result or collections. Although the normal way of
147
+ thinking about application of composed functions is right to left,
148
+ due to how the transduction is built it is applied left to
149
+ right. < code > (compose (tfilter odd?) (tmap sqrt))</ code > will
150
+ create a transducer that first filters out any odd values and then
151
+ computes the square root of the rest.</ p >
149
152
150
153
< h2 id ="state "> State</ h2 >
151
154
@@ -160,6 +163,7 @@ <h2 id="state">State</h2>
160
163
closures, which is efficient and portable, but has all the problems
161
164
of hidden mutable state.</ p >
162
165
166
+
163
167
< h2 id ="naming "> Naming</ h2 >
164
168
165
169
< p > Transducers and procedures that return transducers all have names
@@ -171,15 +175,15 @@ <h2 id="naming">Naming</h2>
171
175
< h2 id ="scope-considerations "> Scope considerations</ h2 >
172
176
173
177
< p > The procedures specified here are only for the collections defined
174
- in R< sup > 7</ sup > RS small. They could easily be extended to support
175
- R< sup > 7</ sup > RS large red docket, but specifying that would require
176
- conforming implementations to also support a substantial part of the
177
- red docket. I therefore leave transduce unspecified for many data
178
- types. It is however encouraged to add [datatype]-transduce for
179
- whatever types your Scheme supports. Adding support for the
180
- collections of the R< sup > 7</ sup > RS red docket (sets, hash-tables,
181
- ilists, rlists, ideque, texts, lseqs, streams and list-queues) is
182
- trivial.</ p >
178
+ in R< sup > 7</ sup > RS small. They could easily be extended to support
179
+ R< sup > 7</ sup > RS large red docket, but specifying that would require
180
+ conforming implementations to also support a substantial part of the
181
+ red docket. I therefore leave transduce unspecified for many data
182
+ types. It is however encouraged to add [datatype]-transduce for
183
+ whatever types your Scheme supports. Adding support for the
184
+ collections of the R< sup > 7</ sup > RS red docket (sets, hash-tables,
185
+ ilists, rlists, ideque, texts, lseqs, streams and list-queues) is
186
+ trivial.</ p >
183
187
184
188
185
189
< h2 id ="lazy-eager "> Eager or lazy semantics</ h2 >
@@ -204,6 +208,7 @@ <h1 id="specification">Specification</h1>
204
208
< h2 id ="applying-transducers "> Applying transducers</ h2 >
205
209
206
210
< h3 id ="list-transduce "> list-transduce</ h3 >
211
+
207
212
< p > < code > (list-transduce</ code > < em > xform f lst</ em > < code > )</ code >
208
213
< br />
209
214
< code > (list-transduce</ code > < em > xform f identity lst</ em > < code > )</ code > </ p >
@@ -222,6 +227,7 @@ <h3 id="list-transduce">list-transduce</h3>
222
227
223
228
224
229
< h3 id ="vector-transduce "> vector-transduce</ h3 >
230
+
225
231
< p > < code > (vector-transduce</ code > < em > xform f vec</ em > < code > )</ code >
226
232
< br />
227
233
< code > (vector-transduce</ code > < em > xform f identity vec</ em > < code > )</ code > </ p >
@@ -253,9 +259,9 @@ <h3 id="port-transduce">port-transduce</h3>
253
259
< code > (port-transduce</ code > < em > xform f init reader port</ em > < code > )</ code > </ p >
254
260
255
261
< p > If < em > port</ em > is provided, it applies < code > (xform f)</ code > to every value
256
- produced by < code > (reader port)</ code > until < code > #eof- object</ code > is returned.
262
+ produced by < code > (reader port)</ code > until the EOF object is returned.
257
263
If < em > port</ em > is not provided, it calls < em > reader</ em > without arguments until
258
- < code > #eof- object</ code > is returned.
264
+ the EOF object is returned.
259
265
</ p >
260
266
261
267
< p > < code > (port-transduce (tfilter odd?) rcons read (open-input-string "1 2 3 4"))</ code >
@@ -269,27 +275,28 @@ <h3 id="generator-transduce">generator-transduce</h3>
269
275
< p > Same as < code > list-transduce</ code > , but for srfi-158-styled generators.</ p >
270
276
271
277
272
-
273
278
< h2 id ="reducers "> Reducers</ h2 >
274
279
275
280
< h3 id ="rcons "> < code > rcons</ code > </ h3 >
276
281
< p > a simple consing reducer. When called without values, it returns
277
- its identity, '(). With one value, which will be a list, it
278
- reverses the list. When called with two values, it conses the
279
- second value to the first.</ p >
282
+ its identity, '(). With one value, which will be a list, it
283
+ reverses the list. When called with two values, it conses the
284
+ second value to the first.</ p >
280
285
281
286
< p > < code > (list-transduce (tmap (lambda (x) (+ x 1)) rcons (list 0 1 2 3))</ code > => < code > (1 2 3 4)</ code > </ p >
282
287
288
+
283
289
< h3 id ="reverse-rcons "> < code > reverse-rcons</ code > </ h3 >
284
- < p > same as < code > rcons</ code > , but leaves the values in their reversed order.</ p >
290
+ < p > same as rcons, but leaves the values in their reversed order.</ p >
285
291
286
292
< p > < code > (list-transduce (tmap (lambda (x) (+ x 1))) reverse-rcons (list 0 1 2 3))</ code > => < code > (4 3 2
287
293
1)</ code > </ p >
288
294
289
295
< h3 id ="rany-pred "> < code > (rany</ code > < em > pred?</ em > < code > )</ code > </ h3 >
296
+
290
297
< p > The reducer version of < code > any</ code > . Returns < code > (reduced
291
298
(pred? value))</ code > if any < code > (pred? value)</ code > returns
292
- non-< code > #f</ code > . The identity is < code > #f</ code > .</ p >
299
+ non-< code > #f</ code > . The identity is < code > #f</ code > .</ p >
293
300
294
301
< p > < code > (list-transduce (tmap (lambda (x) (+ x 1))) (rany odd?) (list 1 3 5))</ code >
295
302
=> < code > #f</ code >
@@ -299,13 +306,15 @@ <h3 id="rany-pred"><code>(rany</code> <em>pred?</em><code>)</code></h3>
299
306
< code > (list-transduce (tmap (lambda (x) (+ x 1))) (rany odd?) (list 1 3
300
307
4 5))</ code > => < code > #t</ code > </ p >
301
308
309
+
302
310
< h3 id ="revery-pred "> < code > (revery</ code > < em > pred?</ em > < code > )</ code > </ h3 >
311
+
303
312
< p > The reducer version of < code > every</ code > . Stops the transduction
304
- and returns < code > (reduced #f)</ code > if any < code > (pred?
313
+ and returns < code > (reduced #f)</ code > if any < code > (pred?
305
314
value)</ code > returns < code > #f</ code > . If every < code > (pred?
306
315
value)</ code > returns true, it returns the result of the last
307
- invocation of < code > (pred? value)</ code > . The identity is
308
- < code > #t</ code > .</ p >
316
+ invocation of < code > (pred? value)</ code > . The identity is
317
+ < code > #t</ code > .</ p >
309
318
310
319
< p > < pre > (list-transduce
311
320
(tmap (lambda (x) (+ x 1)))
@@ -337,19 +346,15 @@ <h3 id="tfilter-pred"><code>(tfilter</code> <em>pred?</em><code>)</code></h3>
337
346
338
347
< h3 id ="tremove-pred "> < code > (tremove</ code > < em > pred?</ em > < code > )</ code > </ h3 >
339
348
< p > Returns a transducer that removes values for which < em > pred?</ em > returns
340
- non-< code > #f</ code > . Must be stateless.</ p >
349
+ non-< code > #f</ code > . Must be stateless.</ p >
341
350
342
351
343
352
< h3 id ="tfilter-map-proc "> < code > (tfilter-map</ code > < em > proc</ em > < code > )</ code > </ h3 >
344
353
< p > The same as < code > (compose (tmap proc) (tfilter values))</ code > .
345
- Must be stateless.</ p >
354
+ Must be stateless.</ p >
346
355
347
356
348
357
< h3 id ="treplace-mapping "> < code > (treplace</ code > < em > mapping</ em > < code > )</ code > </ h3 >
349
- < p > Returns a transducer which checks for the presence of any value passed through it in
350
- < em > mapping</ em > . If a mapping is found, the value of that mapping is
351
- returned, otherwise it just returns the original value.</ p >
352
-
353
358
< p > The argument < em > mapping</ em > is an association list (using equal?
354
359
to compare keys), a hash-table, a one-argument procedure taking
355
360
one argument and either producing that same argument or a
@@ -394,8 +399,8 @@ <h3 id="ttake-while-pred"><code>(ttake-while</code> <em>pred?</em> <em>[retf]</e
394
399
395
400
< h3 id ="tconcatenate "> < code > tconcatenate</ code > </ h3 >
396
401
< p > < code > tconcatenate</ code > < strong > is</ strong > a transducer that
397
- concatenates the content of each value (that must be a list) into
398
- the reduction.</ p >
402
+ concatenates the content of each value (that must be a list) into
403
+ the reduction.</ p >
399
404
400
405
< p > < code > (list-transduce tconcatenate rcons '((1 2) (3 4 5) (6 (7 8) 9)))</ code >
401
406
=>
@@ -415,7 +420,7 @@ <h3 id="tflatten"><code>tflatten</code></h3>
415
420
416
421
417
422
418
- < h3 id ="tdelete-neighbor-dupes "> < code > (tdelete-neighbor-dupes < em > [equality-predicate]</ em > )</ code > </ h3 >
423
+ < h3 id ="tdelete-neighbor-duplicates "> < code > (tdelete-neighbor-duplicates < em > [equality-predicate]</ em > )</ code > </ h3 >
419
424
< p > Returns a transducer that removes any directly following duplicate
420
425
elements. The default < em > equality-predicate</ em > is < code > equal?</ code > .</ p >
421
426
@@ -434,9 +439,9 @@ <h3 id="tremove-duplicates"><code>(tdelete-duplicates <em>[equality-predicate]</
434
439
435
440
< h3 id ="tsegment "> < code > (tsegment</ code > < em > n</ em > < code > )</ code > </ h3 >
436
441
< p > Returns a transducer that groups < em > n</ em > inputs in lists of
437
- < em > n</ em > elements. When the transduction stops, it flushes any
438
- remaining collection, even if it contains fewer than < em > n</ em >
439
- elements.</ p >
442
+ < em > n</ em > elements. When the transduction stops, it flushes any
443
+ remaining collection, even if it contains fewer than < em > n</ em >
444
+ elements.</ p >
440
445
441
446
< p > Stateful.</ p >
442
447
@@ -449,19 +454,22 @@ <h3 id="tpartition-pred"><code>(tpartition</code> <em>pred?</em><code>)</code></
449
454
450
455
< h3 id ="tadd-between-value "> < code > (tadd-between </ code > < em > value</ em > < code > )</ code > </ h3 >
451
456
< p > Returns a transducer which interposes < em > value</ em > between each
452
- value and the next. This does not compose gracefully with
453
- transducers like < code > ttake</ code > , as you might end up ending the
454
- transduction on < em > value</ em > .</ p >
457
+ value and the next. This does not compose gracefully with
458
+ transducers like < code > ttake</ code > , as you might end up ending the
459
+ transduction on < em > value</ em > .</ p >
455
460
456
461
< p > Stateful.</ p >
457
462
458
463
459
464
460
465
< h3 id ="tenumerate-start "> < code > (tenumerate</ code > < em > [start]</ em > < code > )</ code > </ h3 >
461
- < p > Returns a transducer that indexes values passed through it,
462
- starting at < em > start</ em > , which defaults to 0. The indexing is done
463
- through < code > cons</ code > pairs like < code > (</ code > < em > index</ em >
464
- < code > .</ code > < em > input</ em > < code > )</ code > .</ p >
466
+ < p > Returns a transducer that indexes values passed through it,
467
+ starting at < em > start</ em > , which defaults to 0. The indexing is done
468
+ through < code > cons</ code > pairs like < code > (</ code > < em > index</ em >
469
+ < code > .</ code > < em > input</ em > < code > )</ code > .</ p >
470
+
471
+ < p > < code > (list-transduce (tenumerate 1) rcons (list 'first 'second 'third))</ code > =>
472
+ < code > ((1 . first) (2 . second) (3 . third))</ code > </ p >
465
473
466
474
< p > Stateful.</ p >
467
475
@@ -488,7 +496,7 @@ <h3 id="reducedp"><code>(reduced?</code> <em>value</em><code>)</code></h3>
488
496
489
497
490
498
< h3 id ="unreduce "> < code > (unreduce</ code > < em > reduced-container</ em > < code > )</ code > </ h3 >
491
- < p > Returns the value in < em > reduced-container</ em > . </ p >
499
+ < p > Returns the value in < em > reduced-container</ em > </ p >
492
500
493
501
494
502
< h3 id ="ensure-reduced "> < code > (ensure-reduced</ code > < em > value</ em > < code > )</ code > </ h3 >
@@ -497,20 +505,22 @@ <h3 id="ensure-reduced"><code>(ensure-reduced</code> <em>value</em><code>)</code
497
505
498
506
< h3 id ="preserving-reduced "> < code > (preserving-reduced</ code > < em > reducer</ em > < code > )</ code > </ h3 >
499
507
< p > Wraps < em > reducer</ em > in another reducer that encapsulates any
500
- returned reduced value in another reduced container. This is useful
501
- in places where you re-use a reducer with
502
- < em > [collection]-reduce</ em > . If the reducer returns a reduced
503
- value, < em > [collection]-reduce</ em > unwraps it. Unless handled, this
504
- leads to the reduction continuing. </ p >
508
+ returned reduced value in another reduced container. This is useful
509
+ in places where you re-use a reducer with
510
+ < em > [collection]-reduce</ em > . If the reducer returns a reduced
511
+ value, < em > [collection]-reduce</ em > unwraps it. Unless handled, this
512
+ leads to the reduction continuing. </ p >
505
513
506
514
507
515
< h3 id ="list-reduce "> < code > (list-reduce</ code > < em > f identity lst</ em > < code > )</ code > </ h3 >
508
516
< p > The reducing function used internally by < code > list-transduce</ code > . < em > f</ em > is reducer
517
+ as returned by a transducer. < em > i dentity</ em > is the identity (sometimes called "seed") of
509
518
as returned by a transducer. < em > identity</ em > is the identity (sometimes called "seed") of
510
519
the reduction. < em > lst</ em > is a list. If the < em > f</ em > returns a reduced value, the reduction
511
520
stops immediately and the unreduced value is returned.</ p >
512
521
513
522
523
+
514
524
< h3 id ="vector-reduce "> < code > (vector-reduce</ code > < em > f identity vec</ em > < code > )</ code > </ h3 >
515
525
< p > The vector version of < code > list-reduce</ code > .</ p >
516
526
@@ -525,13 +535,12 @@ <h3 id="bytevector-u8-reduce"><code>(bytevector-u8-reduce</code> <em> f identity
525
535
526
536
< h3 id ="port-reduce "> < code > (port-reduce</ code > < em > f identity reader port</ em > < code > )</ code > </ h3 >
527
537
< p > The port version of < code > list-reducer</ code > . It reduces over < em > port</ em > using
528
- < em > reader</ em > until < em > reader</ em > returns < code > #eof- object</ code > .</ p >
538
+ < em > reader</ em > until < em > reader</ em > returns the EOF object.</ p >
529
539
530
540
531
541
< h3 id ="generator-reduce "> < code > (generator-reduce</ code > < em > f identity gen</ em > < code > )</ code > </ h3 >
532
542
< p > The port version of < code > list-reducer</ code > . It reduces over < em > gen</ em > until it returns
533
-
534
- < code > #eof-object</ code > .</ p >
543
+ the EOF object</ p >
535
544
536
545
537
546
< h1 id ="sample-implementation "> Sample implementation</ h1 >
@@ -552,7 +561,7 @@ <h1 id="sample-implementation">Sample implementation</h1>
552
561
553
562
< h1 id ="acknowledgements "> Acknowledgements</ h1 >
554
563
555
- < p > First of all, this would not have been done without Rich Hickey, who
564
+ < p > First of all, this would not have been done without Rich Hickey who
556
565
introduced transducers into Clojure. His talks were important for me
557
566
to grasp the basics of transducers. Then I would like to thank large
558
567
parts of the Clojure community for also struggling with
0 commit comments