<div class=pagetoc> <!-- toc --> </div> *For more workflow docs, see [hledger.org > Workflows](https://hledger.org/workflows.html#more-advanced-workflows). I'm keeping this one here for easier editing; more updates may come. Last updated: 2025-02-01* I am in my 20th year of keeping [PTA](https://plaintextaccounting.org) data. Do my files and process look complicated ? I hope not too much. In essence it's pretty simple, I have: - a bunch of .journal files (organised by year and type of content) - a bunch of .rules files (for importing new transactions from downloaded CSV) - and some handy scripts in Justfile, and bash aliases in bin/bashrc. As usual I am using a journal first setup, which means the journal files are primary and version-controlled, CSV files are secondary and keeping them is optional. This year I am more being more organised about naming CSV and rules files and keeping a few copies of old CSV files for troubleshooting. I am also using more journal sub files and being more uniform and systematic to reduce friction. ## Tools - `hledger` 1.41+ - `hledger-ui` used occasionally, `hledger-web` rarely - `just` for organising task scripts - `git`, `magit` and/or `jujutsu` for version control - `emacs`, `ledger-mode`, `flycheck-hledger` for editing - VS Code for window/context management (with a text emacs in an editor-area terminal tab), and occasionally for editing/search/replace - `rg` for powerful grepping - GNU `sed` for powerful multi-file replace ## Finance directory My hledger files live under the `~/finance` directory. This directory is also a git repository, which helps preserve all file history and makes it easy to inspect or roll back any change. Currently I commit all of the files shown below, except for the .csv files. I use magit and sometimes jujutsu as better UIs for git. In the past I have used a git pre-commit hook to check my files for errors before I can commit. ## Journal files My journal files begin in 2006: ``` 2006 `-- 2006.journal 2007 `-- 2007.journal ... 2025 |-- 2025.journal main file, transactions |-- PEUR25.journal prices for EUR |-- PGBP25.journal prices for GBP |-- PJPY25.journal prices for JPY |-- PVCEB25.journal prices for VCEB |-- accts25.journal account declarations |-- comms25.journal commodity declarations |-- forecast25.journal future transactions |-- iaccts25.journal investment account declarations |-- inv25.journal investment transactions |-- payees25.journal payee declarations (unused) `-- tags25.journal tag declarations all.journal ``` In the beginning I had just one file per year and kept them all in the main directory. But as you can see in 2025 I have many files, so now each year gets its own directory to keep things organised. The year directories are useful for organising other things too, like tax documents. The main journal file in each year is `YYYY.journal`. There's an `all.journal` file at top level, which [includes](https://hledger.org/dev/hledger.html#include-directive) all the `YYYY.journal`s, making all-year reports easy. It also contains account aliases which help normalise my account names over time. ## Journal file organising principles These are the ideas leading to the 2025 file arrangement above. Some may not apply for you, or may not make sense yet, especially if you have less data to manage. - **Simpler is better.**\ Fragmentation adds friction. Complexity is the enemy in PTA. Less is more. - **Separation of concerns.**\ But separate files, each containing one kind of thing, when practical, can make navigation and maintenance easier. - **Livestock not pets.**\ Files containing uniform machine-reproducible data are easier to maintain and evolve than files containing unique hand-crafted text. For efficiency, prefer the former style. (Keeping in mind what `hledger print`, `hledger accounts` etc can preserve and reproduce.) Some of my files contain "pet" data at the top and "livestock" data below, separated by a marker comment. - **Partition by time when you need to.**\ Data will sooner or later need to be partitioned by time period, to keep performance high, avoid clutter, allow evolution, and help with data integrity. Yearly is usually the most natural frequency, so I partition by year. - **Composable periods.**\ Each time slice (each year of data) should be usable by itself, yet ideally also combinable with others. I currently use the [clopen method](https://hledger.org/dev/hledger.html#close) for this, closing AL balances at the end of each year and reopening them at the start of the following year, so that I can freely work with any, all, or some of the year files. `years` and `eachyear` scripts help with the latter. - **Partition by topic only if worthwhile.**\ Accounts and transactions sometimes fall into topical groups which are relatively independent, with few transactions connecting them, and which we tend to work on at different times. In such cases it may be worthwhile to separate these, for a net reduction in cognitive load. For me, it felt right to separate out investment-related transactions, so I have: - general transactions (YYYY.journal, acctsYY.journal, PEUR25.journal..) - investment transactions (invYY.journal, iacctsYY.journal, PVCEB25.journal..) Transactions crossing the boundary between groups (hopefully few) are recorded in both groups, using equity:transfer as a third account. While not strictly needed, this keeps each group balanced and self-contained. - **Standalone subfiles.**\ As with time-based subsets, these topical subfiles should ideally be usable individually, not only when combined. So eg if you are focussed on just the inv25.journal, then reports, `hledger check --strict`, flycheck, etc. should be usable in the usual way. To enable this, invYY.journal includes its own investment accounts and prices files, declares its own investment commodities, and re-declares some general commodities, and the above-mentioned transfer account is used at the boundary. - **Ergonomic filenames.**\ Recognisable, easily typed file names become increasingly important. So: - File names are unique, for clarity when you have multiple years' files open in an editor. - Within each year, file names are unique in their first 1-2 letters, for easy tab-completion and minimal typing. - Price file names begin with P, to group them together and to help keep prefixes unique. - Consistency across years is helpful, so I am gradually reorganising and renaming previous years' files to be more like the 2025 style. ## Main journal I keep the `YYYY.journal` file minimal so that I can clean it up easily. It contains a block of includes, followed by transactions in date order, as printed by `hledger print` and then aligned by Emacs ledger-mode. ```journal include accts25.journal include comms25.journal include payees25.journal include tags25.journal include PEUR25.journal include PGBP25.journal include PJPY25.journal include forecast25.journal 2025-01-01 * opening balances ; clopen:2025 ... ``` I avoid other directives or inter-transaction comment lines, which `print` would not preserve. If I need a comment like that, I give it a date, making a postingless "transaction" that will be preserved and shown in print reports. ## Forecast journal `forecastYY.journal` is a place to note expected future transactions, one-time or recurring. These can be helpful for forecasting and planning, even if they're just estimates. Because I don't want to see them (and have them affect my report dates) all the time, I use (mostly non-recurring) [periodic transaction rules](https://hledger.org/dev/hledger.html#periodic-transactions) instead of ordinary transaction entries. That means these transactions will appear in reports only when I add the [`--forecast` option](https://hledger.org/dev/hledger.html#forecasting). Another way would be to use ordinary transactions, and instead of `include`-ing the forecast journal, add it on the command line when needed, like\ `hledger -f $LEDGER_FILE -f 2025/forecast25.journal print -b today ...` ## Rules files In the rules subdirectory I keep CSV conversion rules for the institutions I import transactions from. ``` rules |-- boi-ichecking.csv.rules |-- common.rules |-- paypal.csv.rules |-- unify-checking.csv.rules |-- unify-savings.csv.rules |-- unify.rules |-- vanguard.csv.rules |-- wf-bchecking.csv.rules |-- wf-bsavings.csv.rules |-- wf-common.rules |-- wf-pchecking.csv.rules |-- wf-psavings.csv.rules ``` ## Archived CSV files Under rules/ there is an archive/ subdirectory, where I archive each CSV file after importing it. You don't need to do this, but I find it useful for troubleshooting or rerunning CSV rules. If you save complete CSV histories in here you could regenerate your journals from them, in principle. ``` |-- archive | |-- boi-6375ae7b-a1dc-4b50-8403-bdf6a05e8cb4.2024-11-22.csv | | ... | |-- boi-c9e13447-7737-43b1-9867-d60555510a56.2025-01-17.csv | |-- paypal.2024-11-19.csv | | ... | |-- paypal.2025-01-07.csv | |-- paypal.2025-01-26.csv | |-- vg-OfxDownload.2024-12-04.csv | | ... | |-- vg-OfxDownload.2025-01-26.csv | |-- vg-OfxDownload.clean.2025-01-26.csv | |-- wf-Checking1.2024-11-19.csv | | ... | |-- wf-Checking1.2025-01-17.csv | |-- wf-Checking1.2025-01-26.csv | |-- wf-Checking2.2024-11-19.csv | | ... | |-- wf-Checking2.2025-01-17.csv | |-- wf-Checking2.2025-01-26.csv | |-- wf-Savings3.2025-01-04.csv | `-- wf-Savings4.2025-01-04.csv ``` ## Downloaded CSV files If I have downloaded CSV files using the web browser but not yet processed them with hledger, they will be waiting in `~/Downloads`. This is where the CSV [`source` rules](https://hledger.org/dev/hledger.html#source) look for new CSV files by default. ``` /Users/simon/Downloads |-- Checking1.csv |-- OfxDownload.csv |-- Savings2.csv ... ``` When downloading bank CSV files, I don't need to care much about which dates I download; hledger's [`import`](hledger.md#import) system will ignore overlaps. I just make sure to download enough data so there's no gaps. I also don't need to care much about managing downloaded CSV files - if I forgot to clean up old downloads, the browser will give the new one a unique filename, and hledger will automatically import the latest one. Now though, I am becoming more disciplined. I have a `just csv` script that "gathers" any new downloaded CSV files, moving them out of `~/Downloads` to the `rules/archive/` directory, with better file names, and importing them. This is a temporary step which I hope to automate this year. ## Accounts Top-level accounts, accounts which need to be declared in a specific order to influence display order, and multiline comments appear at the top of the appropriate accts file. These are followed by a marker comment, and then all the other accounts, one per line, sorted alphabetically. (So these may have at most a single same-line comment.) It's often convenient to append new [account declarations](hledger.md#account-directive) to the appropriate accts file, and re-sort the sortable region at a later time. As a sole proprietor, I practice tracking my business and my personal life as separate entities. Most of the time we are legally one entity, and there are frequent transactions between them, so it's convenient to keep them in the same journal. But at other times it's useful to treat them as separate, and to have the potential of splitting them into completely separate files if needed. So I use a top-level entity prefix to keep each entity's accounts clearly separated. It's always two letters, for less typing, less clutter and good alignment in reports; and upper case for businesses. So my account declarations look like this: ```journal # accts25.journal # business 1 (Joyful Systems) account JS:assets ; type:A account JS:liabilities ; type:L account JS:equity ; type:E account JS:revenues ; type:R account JS:expenses ; type:X account JS:assets:bank ; type:C account JS:assets:cash ; type:C account JS:equity:conversion ; type:V account JS:assets:bank:checking account JS:assets:bank:savings ... # personal account sm:assets ; type:A account sm:liabilities ; type:L account sm:equity ; type:E account sm:income ; type:R, non taxable personal income, eg already-taxed "salary" from JS account sm:revenues ; type:R, taxable personal income account sm:expenses ; type:X account sm:assets:bank ; type:C account sm:assets:cash ; type:C account sm:equity:conversion ; type:V account sm:assets:bank:checking account sm:assets:bank:savings ... ``` While good for clarity, this has always been a bit of a pain, and adds friction when working with other tools. Increasingly I experiment with aliases (defined in an args file or conf file) which either fully merge the two trees, or move the entity name down to level two. %% ## Prices [Market price declarations](#p-directive) are also kept in a separate, included file, `YEARprices.journal`, in date order. New prices can be appended at the end. Eg: ```journal # ... P 2023-06-08 00:00:00 £ $1.25161 P 2023-06-08 00:00:00 ¥ $0.00719 P 2023-06-08 00:00:00 € $1.07676 P 2023-06-09 00:00:00 £ $1.25777 P 2023-06-09 00:00:00 ¥ $0.00717 P 2023-06-09 00:00:00 € $1.07713 P 2023-09-04 00:00:00 € $1.08021 # ... ``` %% %% ## CSV rules Each CSV (or TSV, SSV, ...) data source has a [CSV rules file](hledger.md#csv) declaring how to convert it to meaningful journal entries. Each rules file declares its corresponding data file's name, so that (eg) `hledger import wf-pchecking.csv.rules` will automatically look for the latest-numbered `Checking1*.csv` file in the `~/Downloads` folder. Some rules files are common rules included by the others, eg `wf.rules` and `common.rules`. ## Archived CSV data %% ## Environment variables Official ones, affecting tool behaviour: ``` EDITOR=emacs HLEDGER_UI_EDITOR=vi LEDGER_FILE=/Users/simon/finance/2025/2025.journal ``` A few more that I use locally: ``` HLEDGER_DIR=/Users/simon/finance TIMEDIR=/Users/simon/finance/time TIMEDOT=/Users/simon/finance/time/time-2025.timedot TIMELOG=/Users/simon/finance/time/time-2025.journal ``` The essential one is `LEDGER_FILE`; it should be set to the latest YYYY.journal. Eg `export LEDGER_FILE=~/finance/2025/2025.journal` in your bashrc or other shell config. ## Scripts ``` Justfile main scripts, scripts interface bin additional scripts/tools |-- bashrc bash aliases, functions, env vars |-- cwinv a client-specific wrapper for inv |-- gsheet-csv.hs download CSV from a google sheet |-- inv generic invoice generator |-- paypaljson |-- paypaljson2csv these get CSV from paypal's API `-- wakelog show times of recent wakes/sleeps ``` `Justfile` contains scripts automating my main reports and recurring tasks. `just`[^1] runs these, or lists them to help me remember them. I alias `just` to `j`. Any additional scripts are kept in the `bin` directory, which is added to the shell's PATH. There's also a bashrc file for PTA-specific bash aliases, functions and environment variables, which is sourced by my user bashrc file. The above scripts and an example Justfile can be found at [Scripts and add-ons](scripts.md). Over time, scripts can proliferate and become a maintenance and (human) memory problem. When I want to add a custom script, I think about whether it should be a new recipe in Justfile, an shell alias or function in bashrc, or an standalone script in bin. I recommend using `Justfile` where possible; it is a pretty good combination of ease and power. I used to have many more bash and standalone scripts, but they have mostly been replaced by the Justfile. %% In bin/bashrc I have: ```bash export FINDIR=~/finance export HLEDGER_DIR=$FINDIR export LEDGER_FILE=$FINDIR/2025/2025.journal ``` FINDIR and HLEDGER_DIR are not official, just potentially handy for my own scripts. %% ## Time logs I keep hledger-compatible time logs under finance/ too, in the time/ subdirectory. These are not part of the financial journals above, just stored nearby. ``` time time logs |-- time-200702-200910.timeclock time logged in timeclock format |-- time-2009.journal time logged in journal format | ... |-- time-2025.journal declarations; includes the timedot file |-- time-2025.timedot time logged in timedot format |-- time-all.journal includes all time logs |-- time.journal -> time-2025.journal stable symlinks `-- time.timedot -> time-2025.timedot ``` Related: - [hledger: Timedot](https://hledger.org/hledger.html#timedot) - [hledger: Timeclock](https://hledger.org/hledger.html#timeclock) - [SM's time & task dashboard](https://hledger.org/time-and-task-dashboard.html) %% ## Process In the justfile I have a `foo-import` script for each data source foo, and the `import` script runs all of them. So it's - download one or more CSVs manually in web browser - `j import --dry` - `j import` - load journal in emacs + ledger-mode + flycheck-hledger - select new transactions, `M-q` to align them - for each new transaction: review, refine and mark it cleared when appropriate. - open flycheck's error list with `C-c ! l`, jump to and resolve each issue in turn. These will be things like typos, uncategorised postings, mis-ordered dates etc. - I also run the `recentassertions` check, which periodically forces me to add more recent balance assertions for my main accounts. I do this by adding a "reconcile" transaction (using `yasnippet`, by typing "reconcile" and TAB); filling in the balances hledger expects, to make the errors go away - When the journal is again error-free, I check each account's real-world balance against the "reconcile" transaction's asserted balance (or in a balance report or hledger-ui accounts screen), and resolve any disagreements. - Finally I commit the changes to journal, rules and scripts. Git's `pre-commit` hook runs my checks one more time, catching any last-minute errors. %% --- [^1]: [`just`](just.md) is Like `make`, but better for this purpose. I recommend trying it, even though it doesn't come with your OS; I say this after years of costly battle with make, shell, and other scripting setups.