Merge branch 'main' into someImprovements

Author: TobiasNx

01_Getting_Metafacture.md

@@ -16,5 +16,6 @@ Starting with [Chapter 6](./06_MetafactureCLI.md), we can switch from using Play
 
 To run Metafacture on your local machine you need a Linux/Unix Bash Shell (part of every Linux, MacOS and Windows >=10) with metafacture-core installed. See [Chapter 6](./06_MetafactureCLI.md) for details.
 
+---------------
 
 **Next lesson**: [02 Introduction into Metafacture Flux](./02_Introduction_into_Metafacture-Flux.md)

02_Introduction_into_Metafacture-Flux.md

@@ -101,7 +101,7 @@ Let's start with a link: [https://openlibrary.org/books/OL2838758M.json](https:/
 
 You will see data that look like this:
 
-```JSON
+```json
 {"publishers": ["Belknap Press of Harvard University Press"], "identifiers": {"librarything": ["321843"], "goodreads": ["2439014"]}, "covers": [413726], "local_id": ["urn:trent:0116301499939", "urn:sfpl:31223009984353", "urn:sfpl:31223011345064", "urn:cst:10017055762"], "lc_classifications": ["JA79 .S44 1984", "HM216 .S44", "JA79.S44 1984"], "key": "/books/OL2838758M", "authors": [{"key": "/authors/OL381196A"}], "ocaid": "ordinaryvices0000shkl", "publish_places": ["Cambridge, Mass"], "subjects": ["Political ethics.", "Liberalism.", "Vices."], "pagination": "268 p. ;", "source_records": ["marc:OpenLibraries-Trent-MARCs/tier5.mrc:4020092:744", "marc:marc_openlibraries_sanfranciscopubliclibrary/sfpl_chq_2018_12_24_run01.mrc:195791766:1651", "ia:ordinaryvices0000shkl", "marc:marc_claremont_school_theology/CSTMARC1_barcode.mrc:137174387:3955", "bwb:9780674641754", "marc:marc_loc_2016/BooksAll.2016.part15.utf8:115755952:680", "marc:marc_claremont_school_theology/CSTMARC1_multibarcode.mrc:137367696:3955", "ia:ordinaryvices0000shkl_a5g0", "marc:marc_columbia/Columbia-extract-20221130-001.mrc:328870555:1311", "marc:harvard_bibliographic_metadata/ab.bib.01.20150123.full.mrc:156768969:815"], "title": "Ordinary vices", "dewey_decimal_class": ["172"], "notes": {"type": "/type/text", "value": "Bibliography: p. 251-260.\nIncludes index."}, "number_of_pages": 268, "languages": [{"key": "/languages/eng"}], "lccn": ["84000531"], "isbn_10": ["0674641752"], "publish_date": "1984", "publish_country": "mau", "by_statement": "Judith N. Shklar.", "works": [{"key": "/works/OL2617047W"}], "type": {"key": "/type/edition"}, "oclc_numbers": ["10348450"], "latest_revision": 16, "revision": 16, "created": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "last_modified": {"type": "/type/datetime", "value": "2024-12-27T16:46:50.181109"}}
 ```
 
@@ -136,7 +136,7 @@ inputFile
 
 [When you process the data](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+as-lines%0A%7C+decode-json%0A%7C+encode-yaml%0A%7C+print%0A%3B&data=%7B%22publishers%22%3A+%5B%22Belknap+Press+of+Harvard+University+Press%22%5D%2C+%22identifiers%22%3A+%7B%22librarything%22%3A+%5B%22321843%22%5D%2C+%22goodreads%22%3A+%5B%222439014%22%5D%7D%2C+%22covers%22%3A+%5B413726%5D%2C+%22local_id%22%3A+%5B%22urn%3Atrent%3A0116301499939%22%2C+%22urn%3Asfpl%3A31223009984353%22%2C+%22urn%3Asfpl%3A31223011345064%22%2C+%22urn%3Acst%3A10017055762%22%5D%2C+%22lc_classifications%22%3A+%5B%22JA79+.S44+1984%22%2C+%22HM216+.S44%22%2C+%22JA79.S44+1984%22%5D%2C+%22key%22%3A+%22/books/OL2838758M%22%2C+%22authors%22%3A+%5B%7B%22key%22%3A+%22/authors/OL381196A%22%7D%5D%2C+%22ocaid%22%3A+%22ordinaryvices0000shkl%22%2C+%22publish_places%22%3A+%5B%22Cambridge%2C+Mass%22%5D%2C+%22subjects%22%3A+%5B%22Political+ethics.%22%2C+%22Liberalism.%22%2C+%22Vices.%22%5D%2C+%22pagination%22%3A+%22268+p.+%3B%22%2C+%22source_records%22%3A+%5B%22marc%3AOpenLibraries-Trent-MARCs/tier5.mrc%3A4020092%3A744%22%2C+%22marc%3Amarc_openlibraries_sanfranciscopubliclibrary/sfpl_chq_2018_12_24_run01.mrc%3A195791766%3A1651%22%2C+%22ia%3Aordinaryvices0000shkl%22%2C+%22marc%3Amarc_claremont_school_theology/CSTMARC1_barcode.mrc%3A137174387%3A3955%22%2C+%22bwb%3A9780674641754%22%2C+%22marc%3Amarc_loc_2016/BooksAll.2016.part15.utf8%3A115755952%3A680%22%2C+%22marc%3Amarc_claremont_school_theology/CSTMARC1_multibarcode.mrc%3A137367696%3A3955%22%2C+%22ia%3Aordinaryvices0000shkl_a5g0%22%2C+%22marc%3Amarc_columbia/Columbia-extract-20221130-001.mrc%3A328870555%3A1311%22%2C+%22marc%3Aharvard_bibliographic_metadata/ab.bib.01.20150123.full.mrc%3A156768969%3A815%22%5D%2C+%22title%22%3A+%22Ordinary+vices%22%2C+%22dewey_decimal_class%22%3A+%5B%22172%22%5D%2C+%22notes%22%3A+%7B%22type%22%3A+%22/type/text%22%2C+%22value%22%3A+%22Bibliography%3A+p.+251-260.\nIncludes+index.%22%7D%2C+%22number_of_pages%22%3A+268%2C+%22languages%22%3A+%5B%7B%22key%22%3A+%22/languages/eng%22%7D%5D%2C+%22lccn%22%3A+%5B%2284000531%22%5D%2C+%22isbn_10%22%3A+%5B%220674641752%22%5D%2C+%22publish_date%22%3A+%221984%22%2C+%22publish_country%22%3A+%22mau%22%2C+%22by_statement%22%3A+%22Judith+N.+Shklar.%22%2C+%22works%22%3A+%5B%7B%22key%22%3A+%22/works/OL2617047W%22%7D%5D%2C+%22type%22%3A+%7B%22key%22%3A+%22/type/edition%22%7D%2C+%22oclc_numbers%22%3A+%5B%2210348450%22%5D%2C+%22latest_revision%22%3A+16%2C+%22revision%22%3A+16%2C+%22created%22%3A+%7B%22type%22%3A+%22/type/datetime%22%2C+%22value%22%3A+%222008-04-01T03%3A28%3A50.625462%22%7D%2C+%22last_modified%22%3A+%7B%22type%22%3A+%22/type/datetime%22%2C+%22value%22%3A+%222024-12-27T16%3A46%3A50.181109%22%7D%7D) our book record should now look like this:
 
-```YAML
+```yaml
 ---
 publishers:
 - "Belknap Press of Harvard University Press"
@@ -350,4 +350,6 @@ As you surely already saw I mentioned transform as one step in a metafacture wor
 But aside from changing the serialisation we did not play around with transformations yet.
 This will be the theme of the next session.
 
-Next lesson: [03 Introduction into Metafacture-Fix](./03_Introduction_into_Metafacture-Fix.md)
+---------------
+
+**Next lesson**: [03 Introduction into Metafacture-Fix](./03_Introduction_into_Metafacture-Fix.md)

03_Introduction_into_Metafacture-Fix.md

@@ -35,7 +35,7 @@ Clear it if needed and paste the following Flux in the Flux-File area.
 
 You should end up with something like:
 
-```YAML
+```yaml
 ---
 title: "Ordinary vices"
 ```
@@ -60,7 +60,7 @@ Now let us additionally keep the info that is given in the element `"publish_dat
 
 You should now see something like this:
 
-```YAML
+```yaml
 ---
 title: "Ordinary vices"
 publish_date: "1984"
@@ -80,7 +80,7 @@ Like this.
 
 Fix:
 
-```PERL
+```perl
 retain("title", "publish_date", "notes.value", "type.key")
 ```
 
@@ -102,7 +102,7 @@ retain("title", "publish_date",  "notes.value", "pub_type")
 
 The output should be something like this:
 
-```YAML
+```yaml
 ---
 title: "Ordinary vices"
 publish_date: "1984"
@@ -120,7 +120,7 @@ replace_all("pub_type","/type/","")
 
 If you execute your last workflow with the "Process" button again, you should now see as ouput:
 
-```YAML
+```yaml
 ---
 title: "Ordinary vices"
 publish_date: "1984"
@@ -145,7 +145,7 @@ Comments in Fix start with a hash mark `#`, while in Flux they start with `//`.
 
 Example:
 
-```PERL
+```perl
 # Make type.key a top level element.
 move_field("type.key","pub_type")
 
@@ -171,4 +171,6 @@ Have a look at the fix functions: https://metafacture.org/metafacture-documentat
 or <a href="https://metafacture.org/playground/?flux=%22https%3A//openlibrary.org/books/OL2838758M.json%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-json%0A%7C+fix+%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=move_field%28%22type.key%22%2C%22pub_type%22%29%0Areplace_all%28%22pub_type%22%2C%22/type/%22%2C%22%22%29%0Atimestamp%28%22map_date%22%2Cformat%3A%22yyyy-MM-dd%27T%27HH%3Amm%3Ass%22%2C+timezone%3A%22Europe/Berlin%22%29%0Aretain%28%22title%22%2C+%22publish_date%22%2C+%22by_statement%22%2C+%22pub_type%22%2C+%22map_date%22%29">use timestamp</a>
 </details>
 
-Next lesson: [04 Fix Path](./04_Fix-Path.md)
+---------------
+
+**Next lesson**: [04 Fix Path](./04_Fix-Path.md)

04_Fix-Path.md

@@ -20,14 +20,14 @@ The printed output in YAML format contains a collection of key:value pairs.
 
 There are top level fields like: `title` or `publish_date` which contain only text values or numbers:
 
-```YAML
+```yaml
 ---
 title: "Bullshit Jobs"
 ```
 
 There are also fields like `created` that contain a nested structure where the value is again a key:value pair:
 
-```YAML
+```yaml
 ---
 created:
   type: "/type/datetime"
@@ -37,7 +37,7 @@ created:
 And fields like `source_records[]` do have a list as value:
 
 
-```YAML
+```yaml
 ---
 source_records:
 - "amazon:1501143336"
@@ -63,7 +63,7 @@ For nested objects with deeper structure you add a dot `.` to point to the subfi
 
 If you would have a deeply nested structure like this object:
 
-```YAML
+```yaml
 x:
   y:
     z:
@@ -98,7 +98,7 @@ For YAML and JSON-arrays specifically you also need to use an **array marker** t
 
 In a data set an element sometimes can have multiple instances:
 
-```YAML
+```yaml
 creator: Justus
 creator: Peter
 creator: Bob
@@ -108,11 +108,20 @@ To point to one of the `creator` elements you need to use an index. The first in
 
 If you want to refer to all creators then you can use the `*` sign as a wildcard: `creator.*` refers to all creator elements. The first instance can be selected by the `$first` wildcard and the last by `$last`. This is espacially handy if you do not know how often an element is repeated. When adding an additional repeated element you can use the `$append` or `$prepend` wildcards.
 
-[Append the correct last name to the three investigators: Justus Jonas, Peter Shaw and Bob Andrews. Also prepend "Investigator" to all of them.](https://metafacture.org/playground/?flux=inputFile%0A%7Copen-file%0A%7Cas-records%0A%7Cdecode-yaml%0A%7Cfix%28transformationFile%29%0A%7Cencode-json%28prettyPrinting%3D%22true%22%29%0A%7Cprint%0A%3B&transformation=&data=---%0Acreator%3A+Justus%0Acreator%3A+Peter%0Acreator%3A+Bob%0A)
+[`append` the correct last name to the three investigators: Justus Jonas, Peter Shaw and Bob Andrews. Also `prepend` "Investigator" to all of them.](https://metafacture.org/playground/?flux=inputFile%0A%7Copen-file%0A%7Cas-records%0A%7Cdecode-yaml%0A%7Cfix%28transformationFile%29%0A%7Cencode-json%28prettyPrinting%3D%22true%22%29%0A%7Cprint%0A%3B&transformation=&data=---%0Acreator%3A+Justus%0Acreator%3A+Peter%0Acreator%3A+Bob%0A)
 
 <details>
 <summary>Answer</summary>
+FIX:
+```perl
+append("creator.1"," Jonas")
+append("creator.2"," Shaw")
+append("creator.3"," Andrews")
+prepend("creator.*","Investigator ")
+```
+
 <a href="https://metafacture.org/playground/?flux=inputFile%0A%7Copen-file%0A%7Cas-records%0A%7Cdecode-yaml%0A%7Cfix%28transformationFile%29%0A%7Cencode-json%28prettyPrinting%3D%22true%22%29%0A%7Cprint%0A%3B&amp;transformation=append%28%22creator.1%22%2C%22+Jonas%22%29%0Aappend%28%22creator.2%22%2C%22+Shaw%22%29%0Aappend%28%22creator.3%22%2C%22+Andrews%22%29%0Aprepend%28%22creator.%2A%22%2C%22Investigator+%22%29&amp;data=---%0Acreator%3A+Justus%0Acreator%3A+Peter%0Acreator%3A+Bob%0A">See here</a>
+
 </details>
 
 Hint: Sometimes a repeatable field only can appear only once or not at all. If the record only provide the element once Metafacture (as Catmandu does as well) interpretes the single appearance of an field not a list but as a simple field or an object. You have to adjust your transformation to meet both scenarios. One way how to deal with this is the list bind, which is agnostic to how often an element is provided. The list bind will be introduced in the next session 05.
@@ -134,7 +143,7 @@ source_records:
 
 Our example from above would look like this if creator was a list instead of a repeated field:
 
-```YAML
+```yaml
 creator:
 	- Justus
 	- Peter
@@ -143,7 +152,7 @@ creator:
 
 Lists can be deeply nested if the values are not just strings (list of strings) but objects (list of objects):
 
-```YAML
+```yaml
 characters:
   - name: Justus
     role: Investigator
@@ -155,7 +164,7 @@ characters:
 
 Here is another example:
 
-```YAML
+```yaml
 my:
   colors:
     - black
@@ -209,12 +218,14 @@ XMLs are not just simple elements with key-pair values or objects with subfields
 
 The path for the different attributs and elements are the following:
 
-```YAML
+```yaml
 title.value
 title.type
 title.lang
 ```
 
 If you want to create xml with attributes then you need to map to this structure too. We will come back to lection working with xml in lesson 10.
 
-Next lessons: [05 More Fix Concepts](./05-More-Fix-Concepts.md)
+---------------
+
+**Next lesson**: [05 More Fix Concepts](./05-More-Fix-Concepts.md)