MadMode

Dan Connolly's tinkering lab notebook
wordle of my bookmarks circa 2009
dckc quick notes (May 30, 2025)
  • GnuCash + guile actors for personal finance automation? · dckc/finquick #51 (May 30, 2025)

    Uploaded 100+ un-categorized transactions

    I added a menu item for formatting un-categorized transactions in JSON and POSTing them to the Google Sheet. This works without closing GnuCash.

    Handling redirects was interesting. TIL: a POST switches to a GET on a redirect.

    Next up:

    • match the yarn push-tx-ids functionality by doing transaction matching within the Google Apps Script.
    • do the GnuCash guile / scheme side of pull-transactions

    $ git log10
    2025-05-30 03:03 cb23ab7 feat(gc-sync1): include account code in push-tx-ids
    2025-05-30 03:03 d5bf4c0 feat(sync26): include account code in GnuCash_Uncat
    2025-05-30 02:39 d663549 feat(gc-sync1): POST uncat txs works in 1 case
    2025-05-30 00:43 874f6c9 docs(gc-sync1): trace imports to modules
    2025-05-30 00:06 635f181 feat: format uncategorized splits in JSON

  • GnuCash + guile actors for personal finance automation? · dckc/finquick #51 (May 29, 2025)

    I just about figured out how to test some code using guile, outside GnuCash... but... unlike the python bindings, the scheme bindings don't seem to have a way to load / save a file. ugh!

    $ strings /usr/lib/x86_64-linux-gnu/gnucash/libgnucash-guile.so|grep save
gnc_prefs_get_file_save_compressed
gnc_prefs_set_file_save_compressed
qof_book_session_not_saved
gnc-prefs-get-file-save-compressed
gnc-prefs-set-file-save-compressed
save-window-geometry
hook_save_options
hook_book_saved
qof-book-session-not-saved
    $ strings /usr/lib/x86_64-linux-gnu/gnucash/libgnucash-guile.so|grep load

    2025-05-29 01:05 de87e39 test(gc-sync1): run code with guile, outside gnucash

  • automated python / js refactoring for ocap discipline · dckc/awesome-ocap #55 (May 28, 2025)

    TODO: restore CapPython to working order; use it to give feedback to LLMs.

  • GnuCash + guile actors for personal finance automation? · dckc/finquick #51 (May 26, 2025)

    Assigned 1 category

    ~/projects/finquick/packages/gc-sync1$ git branch --show-current
    gc-sync1
    ~/projects/finquick/packages/gc-sync1$ git log10 -n 4
    2025-05-26 17:37 e73866b feat(sync-uncat): assign category to 1 split
    2025-05-26 17:31 97fb736 feat(gnc-sync1): find uncategorized transactions
    2025-05-26 14:35 822a962 test(gc-sync1): simple-checkbook fixture
    2025-05-26 14:34 1d68a0a feat(gc-sync1): toward category sync from GnuCash menu item

  • CapTP on cloudflare workers? · dckc/awesome-ocap #57 (May 24, 2025)

    JavaScript values are serialized using V8's built-in serializer

    can those values include capability references?

    with the bytes then being embedded in capnp messages for transport.

    Do those messages have something like Payload with content and capTable?

    (btw... @endo/marshal calls them body and slots and has a CapData structure analagous to Payload).

  • blogging analysis paralysis: hakyll, hugo · dckc/madmode-blog #14 (May 24, 2025)

    I kicked the tires...

    ~/gatsby-starter-ghost
18:25 connolly@bldbox$ git remote -v
origin	https://github.com/TryGhost/gatsby-starter-ghost.git (fetch)
origin	https://github.com/TryGhost/gatsby-starter-ghost.git (push)
~/gatsby-starter-ghost
18:25 connolly@bldbox$ git log10 -n 2
2025-05-14 04:26 6894392 Update dependency @tryghost/helpers-gatsby to v2.0.24 (#707)
2025-05-14 00:55 9e8a564 Update dependency @tryghost/helpers to v1.1.93 (#706)

    but then I looked for a "make a new item" button and instead found a ghost config file with an API key.

    Would I use the normal hosted ghost service?
    Where's the version control for the content? Can I give that up?

  • blogging analysis paralysis: hakyll, hugo · dckc/madmode-blog #14 (May 24, 2025)

    somewhat tempting...

  • emacs selection is invisible · dckc/madmode-blog #193 (May 24, 2025)

    haven't seen these symptoms in quite some time

  • trying out brave · dckc/madmode-blog #88 (May 24, 2025)

    I think I meant to close this.

    Meanwhile, it might be time for a "divest from brave" issue:

  • show bitbucket issue activity · dckc/madmode-blog #21 (May 24, 2025)

    I haven't used bitbucket in some time, but I rigged up something close for github comments:

    speaking of bitbucket: Software Heritage archived bitbucket hg repositories before bitbucket took them down

  • local copy of Wikipedia · dckc/madmode-blog #215 (May 24, 2025)

    kiwix packages wikipedia for local use

    https://meta.wikimedia.org/wiki/Data_dump_torrents

    looks like animating it is non-trivial... mediawiki and such

    ah... kiwix...

    magnet link:

    magnet:?&xt=urn:btih:eedb42f768827671b05cef7e22abebf481eb7813&xt=urn:md5:01f14333c83e475d917fb1d5ecff4fb8&xl=109885670576&dn=wikipedia_en_all_maxi_2024-01.zim&as=http%3A%2F%2Fdownload.kiwix.org%2Fzim%2Fwikipedia%2Fwikipedia_en_all_maxi_2024-01.zim&tr=https%3A%2F%2Ftracker.openzim.org%2Fannounce&tr=udp%3A%2F%2Ftracker.openzim.org%3A6969%2Fannounce%0A&ws=http%3A%2F%2Fdownload.kiwix.org%2Fzim%2Fwikipedia%2Fwikipedia_en_all_maxi_2024-01.zim&xs=http%3A%2F%2Fdownload.kiwix.org%2Fzim%2Fwikipedia%2Fwikipedia_en_all_maxi_2024-01.zim.torrent)
  • recent github comments stuck at Feb 15?! · dckc/madmode-blog #227 (May 24, 2025)

    symptoms seem to have gone; I'm not sure why.

    Image

  • blog on talks at RChain devcon in Boulder · dckc/madmode-blog #41 (May 24, 2025)

    fixed in d839d4c

  • xs for js deployment · dckc/madmode-blog #82 (May 24, 2025)

    quickjs seems to support single file deployment.

    I kicked the tires on quickjs today: quickjs-web in finquick 1b85413 but the sqlite integration that I tried seems to be buggy.

  • backups for sandstorm server · dckc/madmode-blog #83 (May 24, 2025)

    I haven't used sandstorm locally for some time.

  • move my sandstorm node to my own domain · dckc/madmode-blog #85 (May 24, 2025)

    I haven't been using sandstorm for some time; I didn't invest much for lack of backups:

  • CapTP on cloudflare workers? · dckc/awesome-ocap #57 (May 24, 2025)

    looks like some point was about a year ago...

  • trade accounting from staketax, osmosis, coinbase via google sheets · dckc/finquick #44 (May 24, 2025)

    106517f was merged

  • GnuCash + guile actors for personal finance automation? · dckc/finquick #51 (May 24, 2025)

    This is an approach to

    I was going to re-scope / narrow that one, but I realize I already have the reduced scope here.

  • use calendar for transaction review · dckc/finquick #2 (May 23, 2025)

    I don't use it these days, but I did get it working.

    let's leave any more write-up out of scope.

  • update clause, views broken due to JSON compare · dckc/finquick #17 (May 23, 2025)

    I'm not using mysql any more.

    in the repo with the data:

    2022-10-04 22:50 32bfbad feat: export sqlite db in ndjson using sqlite-diffable

  • GnuCash incompatible with concurrent access · dckc/finquick #71 (May 23, 2025)
    Brainstorming Approaches to the issue

    ack: "we" = me with a generative AI chatbot. I'm conflicted about the ethics of using this stuff, but I'm trying it out.

    We initially considered several approaches to address the concurrent access challenge:

    1. Application-Level Locking/Coordination: We thought about having Finquick respect GnuCash's gnlock by only performing write operations when the lock was clear, or operating in a read-only mode when GnuCash is active. But GnuCash takes the lock when we open the file and doesn't give it up until we close it.

    2. Automated GnuCash Control: We considered scripting the operating system to close GnuCash before Finquick's "pull" operations and re-open it afterward. While this automates manual steps, it still relies on GnuCash being closed, interrupting the user workflow.

    3. Direct Database Manipulation: We briefly explored directly writing to the underlying database (SQLite, PostgreSQL, MySQL) while GnuCash was open. However, we quickly identified this as highly risky due to GnuCash's in-memory data model and the potential for data corruption and inconsistencies, which GnuCash's gnlock mechanism is designed to prevent.

    4. GnuCash Internal Scripting (Scheme/Python): This emerged as the most promising path to eliminate the stop/start workaround and achieve our goal of continuous data integration. By running scripts within an open GnuCash instance, the script operates within GnuCash's own lock. This allows modifications without external conflicts, leveraging GnuCash's own internal API for data integrity and eliminating the need to stop and start the application. We decided to focus on Scheme due to the user's team's familiarity with it, and its native integration with GnuCash's core.

    Current Workflow Challenges

    To focus the search for solutions, we reflected on a workflow that currently suffers from this issue:

    1. Lossy Bank/Card Ingestion: SheetSync is a service that continuously ingests data from Bank/Card accounts into a Google sheet.
    2. Categorization: We then use the Google sheet to categorize these newly imported transactions.
    3. OFX Import/Sync: SheetSync doesn't reliably reproduce account balances sufficient to reconcile with statements. So we download OFX (or CSV which we convert to OFX) and import into GnuCash.
    4. Push-Txids: We use Finquick to read transaction split guids from GnuCash and push them into the corresponding rows in the spreadsheet. This step works while GnuCash is open.
    5. Pull-Categories: This is the problematic step. We use a finquick script to write category updates (splits) back to GnuCash. This operation currently requires us to close GnuCash to avoid gnlock conflicts and potential data corruption, and then restart GnuCash afterward. This stop/start cycle is the primary inefficiency we aim to eliminate.

    Sometimes we update the db while GnuCash is running and holding the lock; it works most of the time, but we don't want to rely on it.

    For this workflow, internal scripting seems promising.

    Progress on GnuCash Scheme Scripting:

    We embarked on a challenging but fruitful journey to establish a working Scheme scripting environment within GnuCash, which has revealed unique API behaviors along the way.

    Key Scripting Discoveries for Internal GnuCash Scripting:

    1. Confirmed Code Execution & Menu Integration:

      • We initially attempted to use standard gnc:alert or gnc:display-message for a simple "Hello World," but these functions proved unavailable.

      • Discovery: We successfully implemented "Hello World" as a GnuCash Report.

      • Lesson Learned: We used gnc:define-report. We discoverd HTML rendering functions (gnc:html-markup-h1), and the renderer function itself takes only one argument (report-obj). We also found that we must encapsulate content within gnc:make-html-document > gnc:html-document-add-object! > gnc:make-html-text.

      • File Locations: We confirmed that GnuCash looks for configuration in ~/.config/gnucash/config-user.scm and for scripts in ~/.local/share/gnucash/. Our setup uses symlinks to manage these files from a central ~/qtrx/gnucash-config/ directory:

        lrwxrwxrwx 1 connolly connolly 50 May 23 01:39 /home/connolly/.config/gnucash/config-user.scm -> /home/connolly/qtrx/gnucash-config/config-user.scm
lrwxrwxrwx 1 connolly connolly 52 May 23 02:36 /home/connolly/.local/share/gnucash/get-book-info.scm -> /home/connolly/qtrx/gnucash-config/get-book-info.scm
lrwxrwxrwx 1 connolly connolly 59 May 23 02:06 /home/connolly/.local/share/gnucash/hello_gnucash_report.scm -> /home/connolly/qtrx/gnucash-config/hello_gnucash_report.scm

    2. API Naming Inconsistencies & Introspection Limitations:

      • A significant hurdle has been the unpredictable naming conventions of core GnuCash Scheme API functions. For instance, gnc:current-book was unbound, but we found gnc-get-current-book to be the correct function for accessing the current financial book object.

      • Crucially, the standard Scheme bound? function is unavailable, which prevents us from safely checking if a function exists before calling it. This leads to crashes on unbound calls and necessitates a methodical, trial-and-error approach (or direct source code inspection) to discover correct function names.

    3. MAJOR BREAKTHROUGH: We Found xaccSplitSetAccount!

      • Through targeted searching (specifically, an ack search revealing its use in qif-import/qif-to-gnc.scm and other files within /usr/share/guile/site/3.0/gnucash and /usr/share/guile/3.0), we identified the definitive function for modifying a transaction's split account: xaccSplitSetAccount. This function uses an xacc prefix and CamelCase naming, indicating a specific internal API. This is a critical step towards programmatically updating categories.
    Current State:

    We now have a confirmed framework for executing Scheme code within GnuCash
    and accessing the core QofBook object (gnc-get-current-book). Most importantly,
    we know how to update a split's account (xaccSplitSetAccount). This capability is fundamental to automating the category synchronization process.

    This works, modulo gnc-book-get-filename:

    ;; get-book-info-menu.scm
;; A script to get current book info, registered via gnc-add-scm-extension.

(use-modules (gnucash engine))      ; For gnc:current-book, gnc:book-file-name
(use-modules (gnucash app-utils))   ; For gnc:message, if it works for logging.
(use-modules (gnucash core-utils))  ; For N_

;; Define a logging function that tries GnuCash's internal message system first
(define (log-message msg)
        (display (string-append "SCRIPT LOG: " msg "\n"))
        (force-output))

;; This is the function that defines the action for your menu item
(define (run-get-book-info window) ; The lambda in make-menu-item receives the window object
  (let ((book (gnc-get-current-book)))
    (if book
        (let ((filename (gnc-book-get-filename book))) ; <<< TRY THIS: gnc-book-get-filename
          (if filename
              (log-message (string-append "Current GnuCash file: " filename))
              (log-message "Current GnuCash file name could not be retrieved (maybe not saved?).")))
        (log-message "No GnuCash file is currently open."))))

;; Register the menu item
(gnc-add-scm-extension
 (gnc:make-menu-item
  (N_ "Get My Book Info")          ; Name that appears in the menu
  "0d9fe0a6-de1b-4de5-a27c-1919cd9fe484" ; *** IMPORTANT: Generate a NEW unique GUID ***
                                        ; Use 'uuidgen' in a terminal.
  (N_ "Displays info about the currently open GnuCash book") ; Tooltip/Description
  (list (N_ "Tools"))             ; Path: "Tools" menu
  (lambda (window)                      ; The action function when clicked
    (run-get-book-info window))))        ; Call your defined action function
    Next Steps:

    To complete the category-updating script, we plan to identify the precise function names for the remaining core operations within this specific GnuCash build:

    • Finding Transactions: How to retrieve a list of transactions from the QofBook (e.g., to find uncategorized transactions from Finquick).

    • Accessing Splits: How to get the individual splits from a QofTransaction object.

    • Looking Up Accounts: How to find a QofAccount object by its name (for assigning new categories).

    • Saving the Book: The function to persist all changes we make to the script to the GnuCash file.

    We will discover these remaining functions through targeted testing and, if necessary, further inspection of the GnuCash source code's Guile bindings.

  • migrate blog to guix? with Haunt? · dckc/madmode-blog #155 (Mar 29, 2025)

    guix pull

    17:42 connolly@bldbox$ time guix pull
    Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
    Authenticating channel 'guix', commits 9edb3f6 to c31662f (1,991 new commits)...
    ...

    News for channel 'guix'
    Incompatible upgrade of the Syncthing service
    Removable devices now mount under /run/media/$USER' instead of /media'
    Guix System switches to the Shepherd's system log
    Linux-libre updated to 6.13

    real 2m25.610s
    user 2m53.335s
    sys 0m1.385s

  • toot search: when did I discover strudel? · dckc/madmode-blog #211 (Mar 29, 2025)

    mastodon-archive ftw

    https://github.com/kensanata/mastodon-archive

    ~/projects/office-admin/toots$ mastodon-archive text  dckc@social.coop strudel
Dan Connolly @dckc 2024-04-05T04:02:17.052000+00:00
🔗 https://social.coop/@dckc/112216620207497969
found [https://strudel.cc/](https://strudel.cc/) ... started checking it
out... I think an hour flew by

"a new live coding platform to write dynamic music pieces"

    office-admin:

    2025-03-28 18:48 d131c5e feat(toots): mastodon-archive notes

    so... what's left?

    • automatic regular download / sync
    • web ui?
  • automated python / js refactoring for ocap discipline · dckc/awesome-ocap #55 (Feb 15, 2025)

    I think LLMs (and humans) would benefit from a bunch of examples like the airbnb JS style guide:

    // bad
import os

def main():
    os.xyz()

// good
def main(xyz):
    xyz()

if __name__ == '__main__':
    def _script():
        from os import xyz
        main(xyz=xyz)
    _script()

What is Mad Mode?

My day job is writing software to support research at the KU Med Center informatics division, though I'm best known for my work on HTML and Web Architecture at W3C.

I'm a family man, which gives a certain perspective on the KC area, America, and the world we live in.

Between all that, I like to tinker. The bane of my existence is doing things I know the computer could do for me. Have you ever had one of those ideas that won't let go, not even to eat or sleep? My mom said the first time she saw me like that was after they gave me tinker-toys for my 3rd or 4th Christmas. She said I was in "mad scientist mode," just like my father, a chemistry professor.

Mad About...

diigo My Diigo Tags

Acknowledgements