Update doc and docstring

Author: rlan

docs/getting-started/index.md

@@ -4,7 +4,7 @@
 
 Python 3.9, 3.10, 3.11 or 3.12.
 
-These versions are tested via CI/CD. Prior versions of Python may work, just untested.
+All of these versions are tested via [CI/CD]. Due to use of [removeprefix()][], not compatible with previous versions of Python.
 
 ## Installation
 
@@ -48,3 +48,7 @@ Options:
   Note: supported financial institutions are ['ja_bank', 'rakuten_bank',
   'rakuten_card', 'shinsei_bank', 'sumishin_net_bank']
 ```
+
+
+[removeprefix()]: https://docs.python.org/3/library/stdtypes.html#str.removeprefix
+[CI/CD]: https://github.com/rlan/beancount-multitool/actions/workflows/tests.yml

docs/index.md

@@ -15,8 +15,8 @@ The target audience is someone who is quite hands-on with their tools, in the sa
 * The CLI tool saves the user from typing transaction from statements. The tool reads the raw CSV files downloaded from supported financial institutions. It converts the transactions in a CSV file to an importable Beancount file.
     * For example, it can label all debit transactions as `Expenses:JP:Unknown:NameOfInstitution`.
     * and all credit transaction as `Income:JP:Unknown:NameOfInstitution`.
-* The CLI tool uses regular expressions to find recurring expenses. Once found, they are labeled by user-defined account and hashtags, e.g. `Expenses:JP:Food:Grocery` or `Expenses:JP:Food:Restaurant`.
-* Outgoing money transfers can be found via regular expressions and be labeled with a reserved hashtag (`#reconcile`). Then an unique UUID string is generated and appended to that Beancount transaction. This UUID can be used on the receiving Beancount transaction to "link" them for reconcilation.
+* The CLI tool uses regular expressions to find recurring expenses. Once found, they are labeled by user-defined account and tags, e.g. `Expenses:JP:Food:Grocery` or `Expenses:JP:Food:Restaurant`.
+* Outgoing money transfers can be found via regular expressions and be labeled with a reserved tag (`#reconcile`). Then an unique UUID string is generated and appended to that Beancount transaction. This UUID can be used on the receiving Beancount transaction to "link" them for reconcilation.
 
 ### What is it not good for?
 
@@ -38,3 +38,9 @@ If I can only list 3 requirements:
 
 
 [splitting expenses]: https://beancount.github.io/docs/sharing_expenses_with_beancount.html#splitting-expenses
+
+## What are the innovations?
+
+* Parsing the CSV file from each financial instituation to standard data structure, [Pandas Dataframe](https://pandas.pydata.org/pandas-docs/stable/reference/frame.html).
+* [Reserved words](usage/tags.md) in tags injects additional metadata to a Beancount transaction.
+    * I didn't do a survey of tools out there, just a gut feeling.

docs/institutions/ja_bank.md

@@ -32,6 +32,8 @@ column_names = {
 
 Note on dates. On inspection of the CSV file, there are no year in dates, e.g. 4月30日. Thus, the tool uses the current year during conversion. So if you are converting data from last year, edit the generate bean file after conversion.
 
+### Regular expressions
+
 The following two columns are concatentated then passed to regular expressions for matching: `Transaction Classification` and `Description`.
 
 ## Example: label all transactions as default

docs/institutions/rakuten_bank.md

@@ -14,6 +14,8 @@ Header row:
 取引日,入出金(円),取引後残高(円),入出金内容
 ```
 
+### Regular expressions
+
 Regular expressions uses `入出金内容` for matching.
 
 ## Example: label all transactions as default

docs/institutions/rakuten_card.md

@@ -14,6 +14,8 @@ Header row:
 "利用日","利用店名・商品名","利用者","支払方法","利用金額","支払手数料","支払総額","10月支払金額","11月繰越残高","新規サイン"
 ```
 
+### Regular expressions
+
 Regular expressions uses `利用店名・商品名` for matching.
 
 ## Example: label all transactions as default

docs/institutions/shinsei_bank.md

@@ -21,8 +21,12 @@ English header row:
 "Value Date","Description","Debit","Credit","Balance"
 ```
 
+### Regular expressions
+
 Regular expressions uses `摘要` (or `Description`) for matching.
 
+### On sender memo
+
 The raw CSV file does not contain the sender memo that one enters during outgoing money transfers. This info is recorded in the bank's PDF files, instead. My current workaround is manually copying that text from the PDF to the `摘要` (or `Description`) column in the CSV.
 
 ## Example: label all transactions as default

docs/institutions/sumishin_net_bank.md

@@ -14,6 +14,8 @@ Header row:
 "日付","内容","出金金額(円)","入金金額(円)","残高(円)","メモ"
 ```
 
+### Regular expressions
+
 The following two columns are concatentated then passed to regular expressions for matching: `内容` and `メモ`.
 
 ## Example: label all transactions as default

docs/usage/index.md

@@ -10,7 +10,7 @@ A typical workflow using the CLI tool with a Beancount ledger:
 
 ## What to do next?
 
-* For quick start, choose [one](../institutions/index.md) of the financial institutions. Then modify the label-all-as-default example to match your Beacount ledger accounts..
+* For quick start, choose [one](../institutions/index.md) of the financial institutions. Then modify the label-all-as-default example to match your Beacount ledger accounts.
 * For a simple example of using regular expression to label accounts, see the regular expression example for [JA Bank](../institutions/ja_bank.md).
 * Learn about [configurations](configs.md) of `bean-mt` CLI tool.
 * Here are some [examples](examples.md) of common usage.

docs/usage/tags.md

@@ -0,0 +1,30 @@
+# Tags
+
+A generic Beancount transaction looks like this:
+
+```txt
+{date} * "{payee}" "{narration}" {" ".join({tags})}
+  for key, value in {metadata}:  # aka transaction metadata
+    {key}: "{value}"
+  {source_account}
+  {flag}{account}  {amount} {currency}
+    for key, value in {account_metadata}:  # aka account metadata
+      {key}: "{value}"
+```
+
+Tags are user input. Here we define a list of reserved words for tags that injects additional metadata.
+
+* `#reconcile`
+    * Adds a (random) uuid string to a `uuid` key to transaction metadata of a debit transaction (amount is negative). Example:
+
+    ```txt
+    uuid: "603cd901-a734-45d0-a100-dc01c22b5b4b"
+    ```
+
+    * Adds a empty string to a `uuid` key to transaction metadata of a credit transaction (amount is positive). Example:
+
+    ```txt
+    uuid: ""
+    ```
+
+    This is to reconcile duplicate transactions for a money transfer between two accounts. My personal convention is to keep the debit one, copy the UUID, paste the UUID to the credit one and comment the credit one.

mkdocs.yml

@@ -36,6 +36,7 @@ nav:
   - getting-started/index.md
   - Usage:
     - usage/index.md
+    - usage/tags.md
     - usage/configs.md
     - usage/examples.md
   - Financial Institutions:

src/beancount_multitool/as_transaction.py

@@ -24,12 +24,12 @@ def as_transaction(
     str
         A beancount transaction entry:
 
-        {date} * "{payee}" "{narration}" {" ".join(tags)}
-          for key, value in metadata:
+        {date} * "{payee}" "{narration}" {" ".join({tags})}
+          for key, value in {metadata}:
             {key}: "{value}"
           {source_account}
           {flag}{account}  {amount} {currency}
-            for key, value in metadata:
+            for key, value in {account_metadata}:
               {key}: "{value}"
     """
     # Add "#" prefix if does not exist

src/beancount_multitool/get_value.py

@@ -16,6 +16,11 @@ def get_value(config: dict, section: str, variable: str) -> str:
     -------
     str
         value
+
+    Raises
+    ------
+    KeyError
+        if "section" key or "variable" key does not exist.
     """
     if section in config:
         if variable in config[section]:

tests/data/ja_bank/README.md

@@ -0,0 +1,13 @@
+# Read Me
+
+A test case that has no mappings defined, so all transactions are mapped to the default.
+
+Command:
+
+```sh
+bean-mt \
+  ja_bank \
+  ./config.toml \
+  ./test.csv \
+  --output output.bean
+```

tests/data/rakuten_bank/README.md

@@ -0,0 +1,13 @@
+# Read Me
+
+A test case that has no mappings defined, so all transactions are mapped to the default.
+
+Command:
+
+```sh
+bean-mt \
+  rakuten_bank \
+  ./config.toml \
+  ./test.csv \
+  --output output.bean
+```

tests/data/rakuten_card/README.md

@@ -0,0 +1,13 @@
+# Read Me
+
+A test case that has no mappings defined, so all transactions are mapped to the default.
+
+Command:
+
+```sh
+bean-mt \
+  rakuten_card \
+  ./config.toml \
+  ./test.csv \
+  --output output.bean
+```

tests/data/shinsei_bank/README.md

@@ -0,0 +1,13 @@
+# Read Me
+
+A test case that has no mappings defined, so all transactions are mapped to the default.
+
+Command:
+
+```sh
+bean-mt \
+  shinsei_bank \
+  ./config.toml \
+  ./test.csv \
+  --output output.bean
+```

tests/data/sumishin_net_bank/README.md

@@ -0,0 +1,13 @@
+# Read Me
+
+A test case that has no mappings defined, so all transactions are mapped to the default.
+
+Command:
+
+```sh
+bean-mt \
+  suminshin_net_bank \
+  ./config.toml \
+  ./test.csv \
+  --output output.bean
+```

tests/data2/ja_bank/README.md

@@ -1,6 +1,6 @@
 # Read Me
 
-A test case that demostrates a reserved hashtag `#reconcile`. This test case also demonstrates multiple account matches for one expense.
+A test case that demostrates a reserved tag `#reconcile`. This test case also demonstrates multiple account matches for one expense.
 
 Command:
 
@@ -9,5 +9,5 @@ bean-mt \
   ja_bank \
   ./config.toml \
   ./test.csv \
-  --output out.bean
+  --output output.bean
 ```