smalltalk programming for the web

Show Me The Money

28 August 2016

I wrote a money class on Pharo 1.x a few years back. Didn't use it beyond simple calculations as my financial computing needs are met using a spreadsheet-and-wiki combo. I've now reached the limits of that approach so looking to restart my business/finance-oriented programming with Pharo.

Here are other money-related implementations for Pharo/Squeak that I found:

Money - "The money package from Squeaksource. Originally from Smalltalk Best Practice Patterns." Most recent commit in Apr 2016. Installs cleanly on Pharo 5. 10 of 11 tests pass. The failed test looks like a minor bug. Older versions exist on Squeaksource.

Aconcagua-Money - "Aconcagua units for common domains - Time, Money, and Weight at the time of this writing." Most recent commit in May 2015. Installs cleanly on Pharo 5 with Aconcagua already installed. Has no test. Presumably relies on Aconcagua's own tests, as the classes are small and look simple.

FixedDecimal - "A FixedDecimal is similar to a ScaledDecimal, but different in certain select ways. It's (sic) primary purpose was to be able to represent precise decimals for such things as representing money - where ScaledDecimals leave something to be desired. [...] Forked from Credits to Chris Cunningham." Most recent commit in Dec 2013. 11 of 12 tests pass. My money class uses FixedDecimal.

ScaledDecimal - Part of the image.

SmallPOS - "Framework for fast building of accounting, trading (including Points of Service), CRM and so on software. Based on GLORP, Magritte and Seaside." Contains a class CMACurrency.

Money Talk

From Kent Beck, here, "Re: Money in particular, I have implemented this one five times. The only intersection of the five implementations is a Money object whose amount is some kind of number, and which has a currency. Everything else is different. [...] Therefore, I carry around in my head a pattern language for creating Money and I reimplement it every time from scratch."

Automating web content deployment with Fossil

12 August 2016

I've migrated this site to Pharo 5 from Pharo 3. One notable behind-the-scene change is that content is now deployed using Fossil, the "simple, high-reliability, distributed software configuration management system". Fossil is usually compared with DVCS such as Git and Mercurial.

This server runs the published version of this site. (Of course.) The development version is on my laptop. Now, when I wish to publish changes from my development version, say, when I have written a new blog post, I 'fossil commit', then 'fossil push', whereupon the new blog post and associated artefacts such as images are deployed into the published version on this server.

There are any number of articles on the web describing using Git hooks to automate deploying web content. In this post, I describe using Fossil to do the same. Some familiarity with Fossil is assumed.


To begin, build Fossil with TH1 hooks and Tcl integration. Deploy it both locally and on the server, by copying the single Fossil executable to appropriate directories.

Create a new repostory locally. Take note of the admin user ID and generated password. Let's assume that the admin user ID is 'laptop-user'. Be aware that Fossil has its own user IDs, passwords and roles, distinct from the underlying operating system's.

laptop% mkdir ~/repo
laptop% cd ~/repo
laptop% fossil init webtree.fossil
project-id: 3c05c3016eeabf8e87816ee218c6a86d3c87b950
server-id:  ff42bc86dba1a26b1d94b64685f7c09d02581617
admin-user: laptop-user (initial password is "1fe2ff")

Check out the repository locally:

laptop% mkdir ~/webtree
laptop% cd ~/webtree
laptop% fossil open ~/repo/webtree.fossil

Add new content:

laptop% cd ~/webtree
laptop% echo "Hello, world, from Fossil." > helloworld.txt
laptop% fossil add helloworld.txt 
laptop% fossil commit -m "First commit." 

Upon commit, the repository, ~/repo/webtree.fossil, is updated. Note that the repository is a single file that can be copied, renamed, moved to other computers, etc. Let's scp it to our server and set up the following directory structure there. Here I use /x to keep path names short for this post.

    fossil         <= The Fossil executable.
    fossilupdate   <= Shell script, described below.
  webtree/         <= Checked out tree of webtree.fossil.
    webtree.fossil <= The repository.

To populate /x/webtree:

server% cd /x/webtree
server% fossil open /x/repo/webtree.fossil
server% ls -a
./  ../  .fslckout  helloworld.txt

The file .fslckout is Fossil's checkout database.

Create the shell script /x/bin/fossilupdate:

server% cd /x/bin
server% cat > fossilupdate
cd /x/webtree
/x/bin/fossil update
server% chmod 755 fossilupdate

Run the Fossil server. Here I use port 8081.

server% cd /x/webtree
server% /x/bin/fossil server --port 8081
Listening for HTTP requests on TCP port 8081

With your web browser, navigate to the Fossil server and login using the admin user ID and password you noted down earlier. Go to Admin > Settings. Disable autosync, enable tcl, enable th1-hooks, and click 'Apply Changes'. Go to Admin > Transfers > Push. Enter below command into the text box and click 'Apply Changes'.

tclInvoke exec /x/bin/fossilupdate & 

This sets up the Fossil transfer hook, which will fire after your Fossil server processes a 'fossil push' request.

Back on the laptop, commit another file:

laptop% cd ~/webtree
laptop% echo "Hello again." > take2.txt
laptop% fossil add take2.txt
laptop% fossil commit -m "Take 2."
laptop% fossil push http://laptop-user@server:8081/
password for laptop-user: 
remember password (Y/n)? n
Round-trips: 2   Artifacts sent: 2  received: 0
Push done, send: 1075  received: 1199  ip: <server>

The push from the laptop will trigger the abovementioned transfer hook configured in the server's Fossil server instance, which will update /x/webtree on the server. Viola!

Automatically starting the Fossil server

It is often desirable to start the Fossil server automatically upon server startup. On my server, I use daemontools. However, for some reason daemontools' setuidgid program is unable to run 'fossil server' in the correct directory. So I use Ubuntu's start-stop-daemon instead, and this is the daemontools run file:

exec /sbin/start-stop-daemon --start \
  -c cms:cms -d /x/webtree \
  --exec /x/bin/fossil \
  -- server --port 8081

Create a new user, say, 'cms', and set up ownership of /x:

User:Group   Dir/File
root:root    /x/
root:root      bin/
root:root        fossil
root:root        fossilupdate
cms:cms        webtree/
cms:cms        repo/
cms:cms          webtree.fossil
root:root        run <= daemontools run file

Link /x/repo into daemontools, and 'fossil server' runs as the 'cms' user in /x/webtree.


/x/webtree/.fslckout is Fossil's checkout database. If your web server serves content directly from /x/webtree, you should configure it to not serve the .fslckout file.

An alternate practice (according to those Git hook articles) is to rsync the content of /x/webtree to another directory, and it is this second directory that the web server reads from. In such a case, it is still necessary to avoid rsync'ing the .fslckout file.

Users in Fossil

As mentioned, Fossil maintains its own user IDs, passwords, and roles. In this post, I assumed that the Fossil admin user is called 'laptop-user' and used it for 'fossil push'. It is preferable to set up a separate lower privileged Fossil user and use that instead.

Glorp-SQLite3 for Pharo 5

6 June 2016

I'm pleased to announce the release of Glorp-SQLite3 for Pharo 5.

Developed and tested on Linux. Known working on Windows 7. Your Pharo 5 VM needs to be able to find or the Windows equivalent.

Take a fresh Pharo 5 image. Run the following:

Gofer it
    smalltalkhubUser: 'TorstenBergmann' project: 'UDBC';
(Smalltalk at: #ConfigurationOfUDBC) loadBleedingEdge.

Gofer it
    smalltalkhubUser: 'DBXTalk' project: 'Glorp';
    configurationOf: 'Glorp';
#ConfigurationOfGlorp asClass project stableVersion load.

Gofer it
    smalltalkhubUser: 'DBXTalk' project: 'Glorp';
    package: 'Glorp-SQLite3';

GlorpSQLite3CIConfiguration new configureSqlite3.
GlorpDemoTablePopulatorResource invalidateSetup.

Run the Glorp tests in TestRunner. All tests should pass, with 12 tests skipped. The database file is sodbxtestu.db in your image directory.

SQLite and JSON

24 April 2016

SQLite 3.9.0, released in Oct 2015, introduced the json1 extension that provides functions to manage JSON content stored in an SQLite database. The extension source code is included in the SQLite source amalgamation and is enabled by the -DSQLITE_ENABLE_JSON1 option.

The following assumes that the json1 functionality is available. As sample data, I use the stocks data set from Here's what the wrapped first line of the JSON data file looks like partially:

{ "_id" : { "$oid" : "52853800bb1177ca391c17ff" }, "Ticker" : "A", "Profit
Margin" : 0.137, "Institutional Ownership" : 0.847, "EPS growth past 5
years" : 0.158, "Total Debt/Equity" : 0.5600000000000001, "Current Ratio" :
3, "Return on Assets" : 0.089, "Sector" : "Healthcare", "P/S" : 2.54,
"Change from Open" : -0.0148, "Performance (YTD)" : 0.2605, "Performance
(Week)" : 0.0031, "Quick Ratio" : 2.3, "Insider Transactions" : -0.1352,

To load the file into an SQLite database using NBSQLite3:

| jr db |
jr := (FileSystem / 'tmp' / 'stocks.json') asFileReference readStream.
jr ascii.
db := NBSQLite3Connection openOn: '/tmp/s.db'.
[   db execute: 'create table s (d json)'.
    db execute: 'begin'.
    [ jr atEnd ] whileFalse: [ 
        db execute: 'insert into s values (json(?))' "<== Note."
            with: (Array with: jr nextLine) ] 
    db execute: 'commit'.
] ensure: [ db close ]
jr close.

Note the invocation of the json() SQL function, which verifies that its argument is a valid JSON string and returns a minified version of that JSON string.

Let's inspect the created database with the sqlite3 shell tool:

$ wc -l stocks.json
6756 stocks.json
$ sqlite3 s.db
SQLite version 3.9.2 2015-11-02 18:31:45
Enter ".help" for usage hints.
sqlite> select count(*) from s;
sqlite> .schema
CREATE TABLE s (d json);

The json_extract(X, P1, P2, ...) function extracts and returns one or more values from the JSON structure X:

sqlite> select json_extract(d, '$.Sector', '$.Company') 
   ...> from s 
   ...> where json_extract(d, '$.Ticker') = 'AAPL';
["Consumer Goods","Apple Inc."]

The json_each() table-valued function walks the JSON value provided as its argument and returns one row for each element. The schema for the table returned by json_each() is as follows:

CREATE TABLE json_tree(
    key ANY,             -- key for current element relative to its parent
    value ANY,           -- value for the current element
    type TEXT,           -- 'object','array','string','integer', etc.
    atom ANY,            -- value for primitive types, null for array & object
    id INTEGER           -- integer ID for this element
    parent INTEGER,      -- integer ID for the parent of this element
    fullkey TEXT,        -- full path describing the current element
    path TEXT,           -- path to the container of the current row
    json JSON HIDDEN,    -- 1st input parameter: the raw JSON
    root TEXT HIDDEN     -- 2nd input parameter: the PATH at which to start

Here's json_each() in action:

sqlite> select key, value 
   ...> from s, json_each(s.d) 
   ...> where json_extract(d, '$.Ticker') = 'AAPL'
   ...> limit 10;
Profit Margin|0.217
Institutional Ownership|0.621
EPS growth past 5 years|0.424
Total Debt/Equity|0.14
Current Ratio|1.7
Return on Assets|0.186
Sector|Consumer Goods

Let's run the same query in Pharo:

| q db v |
q := 'select key, value from s, json_each(s.d) where json_extract(d, "$.Ticker") = "AAPL" limit 10'.
db := NBSQLite3Connection openOn: '/tmp/s.db'.
[   (db execute: q) rows do: [ :r |
        Transcript show: (r at: 'key') asString, ' = '.
        v := r at: 'value'.
        Transcript show: v asString, ' (', v class asString, ')'; cr ]
] ensure: [ db close ]

Transcript displays thusly. As we can see, SQLite and NBSQLite3 handle numerical values in the JSON data properly.

_id = {"$oid":"52853800bb1177ca391c1809"} (ByteString)
Ticker = AAPL (ByteString)
Profit Margin = 0.217 (Float)
Institutional Ownership = 0.621 (Float)
EPS growth past 5 years = 0.424 (Float)
Total Debt/Equity = 0.14 (Float)
Current Ratio = 1.7 (Float)
Return on Assets = 0.186 (Float)
Sector = Consumer Goods (ByteString)
P/S = 2.75 (Float)

Notice that the first row is well-formed JSON. If we use json_tree() instead of json_row(), that JSON structure will be returned as a key/value row as well.

Finally, let's read the first JSON record using NeoJSON:

| q db  |
q := 'select json_extract(d, "$") as data from s limit 1'.
db := NBSQLite3Connection openOn: '/tmp/s.db'.
[   (NeoJSONReader on: ((db execute: q) rows first at: 'data') readStream)
        next explore
] ensure: [ db close ].

This is powerful stuff. Well worth a look.