fpwebview updates

06 Apr 2024

I've published latest updates to fpwebview:

  • library files for x86_64 for Linux, macOS and Windows
  • library files for i386 for Windows
  • library file for aarch64 for macOS
  • built from latest webview (commit dated Feb 2024)
  • built from WebView2 1.0.2365.46 (published Feb 2024)
  • demos updated to match changes in webview
  • LCL embedding demo tested with Lazarus 3.2
Tags: fpwebview, Pascal

Using mORMot with Azure Cosmos DB MongoDB

04 Mar 2023

mORMot is an "open source client-server ORM SOA MVC framework" for Delphi and FPC. Version 2 was recently released. Typical of such frameworks, mORMot supports many database drivers for connection to various well-known database products, one of which is MongoDB, for which mORMot provides a native wire protocol implementation in Pascal.

Microsoft Azure Cosmos DB is a brand of cloud-hosted database-as-a-service, a.k.a DBaaS. Cosmos DB supports a number of APIs / data models, or, in plain language, it provides hosted PostgreSQL, MongoDB, and several other products. For a few years now, Microsoft has been offering Try Azure Cosmos DB free, good for 30 days, with ability to extend or convert to paid account.

In this and subsequent posts, we'll look at using mORMot with Azure Cosmos DB.

First, sign up with Azure Cosmos DB, then choose to create an Azure Cosmos DB for MongoDB. When done, follow the link to the Azure portal to manage the database instance. Take note of the Settings menu on the side bar on the left.

Azure Cosmos DB Settings

Click on Connection strings to get the information required to connect to our MongoDB instance. Take note of HOST, PORT, USERNAME and PRIMARY PASSWORD fields in the default Read-write Keys tab. Note that SSL (more accurately, TLS) is mandatory, for obvious reasons.

Azure Cosmos DB Connection String

Next, we will modify mORMot's bundled MongoDB example minimally to use our Azure MongoDB instance. In MongoDBTests.dpr, add mormot.lib.openssl11 to the uses clause. All other changes will be made to MongoDBTestCases.pas; below, all line numbers refer to the original source file:

  • Line 11, define TESTMONGOAUTH.

  • Lines 39 and 41, change MONGOSERVER and MONGOPORT to the connection string's HOST and PORT values.

  • Line 181, change DB_NAME to 'mormot'.

  • Lines 182 and 183, change USER_NAME and USER_PWD.

  • Comment out lines 208-218. Instead of creating a new MongoDB user, we'll use the pre-created user for this demo.

  • Change line 220 and add following lines to enable TLS:

  fClient := TMongoClient.Create(MONGOSERVER, MONGOPORT, [mcoTls]); // This is line 220
  with fClient.ConnectionTlsContext do
    begin
      Enabled := true;
      IgnoreCertificateErrors := false;
    end;
  • Change line 233 (remember, line number refers to original file) to following:
  fDB := fClient.OpenAuth(DB_NAME, USER_NAME, USER_PWD, true); 
  • Change line 462 and add following lines to enable TLS:
  fMongoClient := TMongoClient.Create(MONGOSERVER, MONGOPORT, [mcoTls]); // This is line 462
  with fMongoClient.ConnectionTlsContext do
    begin
      Enabled := true;
      IgnoreCertificateErrors := false;
    end;
  • Change line 472 (again, this is line number in original file) to following:
  fDB := fMongoClient.OpenAuth(DB_NAME, USER_NAME, USER_PWD, true); 

And that's it. I have posted the complete set of files required to build this demo to my mormot-cloud repo.

Rebuild and run:

% lazbuild MongoDBTests.lpi
...
(1008) 2758 lines compiled, 1.1 sec, 3989044 bytes code, 2500148 bytes data
(1022) 3 hint(s) issued
(1023) 1 note(s) issued

% ./MongoDBTests

   Mongo DB
  ----------


1. Direct access

 1.1. Direct: 
  - Connect to local server: 6 assertions passed  3.25s
  - Drop and prepare collection: 6 assertions passed  239.51ms
  - Fill collection: 353 assertions passed  33.41s
     100 rows inserted in 32.40s i.e. 3/s, aver. 324.03ms, 0.9 KB/s
  - Drop collection: no assertion  510.53ms
  - Fill collection bulk: 2 assertions passed  2.56s
     100 rows inserted in 2.25s i.e. 44/s, aver. 22.51ms, 4.3 KB/s
  - Graceful reconnect: 3 assertions passed  2.16s
  - Read collection: 603 assertions passed  3m07
     100 rows read at once in 892.18ms i.e. 112/s, aver. 8.92ms, 9.1 KB/s
  - Update collection: 136 assertions passed  1m01
     100 rows updated in 30.67s i.e. 3/s, aver. 306.79ms, 793 B/s
  - Delete some items: 82 assertions passed  31.43s
     20 rows deleted in 6.37s i.e. 3/s, aver. 318.57ms, 514 B/s
  Total failed: 0 / 1,191  - Direct PASSED  5m22


2. ORM

 2.1. ORM: 
  - Connect to local server: 6 assertions passed  3.13s
  - Insert: 102 assertions passed  33.87s
     100 rows inserted in 32.95s i.e. 3/s, aver. 329.58ms, 755 B/s
  - Insert in batch mode: 104 assertions passed  2.14s
     100 rows inserted in 1.84s i.e. 54/s, aver. 18.47ms, 7 KB/s
  - Retrieve: 1,001 assertions passed  31.26s
     100 rows retrieved in 31.26s i.e. 3/s, aver. 312.62ms, 1.3 KB/s
  - Retrieve all: 901 assertions passed  325.37ms
     100 rows retrieved in 325.32ms i.e. 307/s, aver. 3.25ms, 39.8 KB/s
  - Retrieve one with where clause: 1,000 assertions passed  31.63s
     100 rows retrieved in 31.63s i.e. 3/s, aver. 316.37ms, 1.3 KB/s
  - Retrieve from SQL: 4,709 assertions passed  8.30s
     535 rows retrieved in 8.30s i.e. 64/s, aver. 15.52ms, 8.2 KB/s
  - Update: 902 assertions passed  31.29s
     100 rows updated in 30.98s i.e. 3/s, aver. 309.81ms, 0.9 KB/s
  - Blobs: 2,707 assertions passed  2m03
     100 rows updated in 30.99s i.e. 3/s, aver. 309.96ms, 638 B/s
  - Delete: 843 assertions passed  8.89s
     20 rows deleted in 5.52s i.e. 3/s, aver. 276.01ms, 480 B/s
  - Delete in batch mode: 743 assertions passed  544.53ms
     20 rows deleted in 307.30ms i.e. 65/s, aver. 15.36ms, 1 KB/s
  Total failed: 0 / 13,018  - ORM PASSED  4m35

Using MongoDB 3.6.0 
Running on Ubuntu 20.04.5 LTS - Linux 5.4.0-144-generic
Compiled with mORMot 2.0.4952
Generated with: Free Pascal 3.3.1 64 bit Linux compiler

Time elapsed for all tests: 9m57
Performed 2023-03-04 17:24:04 by pierce

Total assertions failed for all test suits:  0 / 14,209
! All tests passed successfully.

Memory Usage Report:
 Flags: SERVER  assumulthrd erms debug repmemleak
 Small:  2K/200KB  including tiny<=128B arenas=8 fed from Medium
 Medium: 3MB/3MB    peak=3MB current=3 alloc=3 free=0 sleep=0
 Large:  0B/0B    peak=0B current=0 alloc=0 free=0 sleep=0
 Small Blocks since beginning: 108K/9MB (as small=43/46 tiny=55/56)
  48=30K  32=27K  64=10K  128=8K  112=7K  192=4K  80=4K  240=2K
  96=2K  176=2K  144=2K  160=1K  224=896  208=700  528=669  256=468
 Small Blocks current: 2K/200KB
  32=1K  48=689  352=176  64=81  112=70  128=68  80=41  96=38
  880=7  416=7  160=6  192=6  624=4  224=4  1264=3  576=3

%

Took about 10 minutes to run. Not very fast... To be fair, default region for Try Cosmos DB for free is North America, and I am in Southeast Asia.

Go back to Azure portal, click on Data Explorer, and marvel at the data just inserted:

Azure Cosmos DB MongoDB Data Explorer

In a subsequent post we shall see how to avoid hardcoding our database password in the source code.

Tags: Azure, MongoDB, mORMot, Pascal

Changing data root directory in rootless Docker

22 Feb 2023

I switched my Linux laptop to rootless Docker recently.

Running as root, by default Docker stores its data in /var/lib/docker. Switching over to rootless Docker, the equivalent location is now under my home directory.

My laptop is a two-disk setup (see my write up on dual booting Windows 10 and Xubuntu), with separate LVM volume groups (VGs) for home, var and tmp. The largest usage of var was due to Docker. With rootless Docker, I still want Docker to use var and not my home directory for its stuff.

I had set up rootless Docker with systemd, as recommended by Docker documentation, which also says:

  • The data dir is set to ~/.local/share/docker by default.
  • The daemon config dir is set to ~/.config/docker by default.

With systemd, rootless Docker's config file is in ~/.config/systemd/user/docker.service. The first three lines of the service stanza in that file looks like this:

[Service]
Environment=PATH=<blah blah blah>
ExecStart=/bin/dockerd-rootless.sh
ExecReload=/bin/kill -s HUP $MAINPID

Line 2, ExecStart tells us that rootless Docker is executed by /bin/dockerd-rootless.sh, which, by naming convention, is a shell script. And, helpfully, the comment block at the top of that file tells what it does:

#!/bin/sh
# dockerd-rootless.sh executes dockerd in rootless mode.
#
# Usage: dockerd-rootless.sh [DOCKERD_OPTIONS]

So this script, /bin/dockerd-rootless.sh, takes DOCKERD_OPTIONS. And what might those be? Docker's documentation says:

Usage: dockerd COMMAND

A self-sufficient runtime for containers.

Options:
      ...<options in alphabetical order>...
      --data-root string                      Root directory of persistent Docker state (default "/var/lib/docker")

Aha! Putting it together, the way to set data-root directory for rootless Docker is to modify the ExecStart key in ~/.config/systemd/user/docker.service, like this:

[Service]
Environment=PATH=<blah blah blah>
ExecStart=/bin/dockerd-rootless.sh --data-root /var/lib/docker-1000
ExecReload=/bin/kill -s HUP $MAINPID

As root, I created /var/lib/docker-1000 and then chown 1000:1000 it, to serve as my rootless Docker setup's data root directory. Restarted rootless Docker, and it now uses the new data root directory:

% systemctl --user stop docker
% systemctl --user daemon-reload
% systemctl --user start docker
% docker info | egrep "Root Dir"
 Docker Root Dir: /var/lib/docker-1000
Tags: Docker

TIL: exFAT, macOS and Windows 10

11 Jan 2023

TIL:

  • USB drive formatted as exFAT by macOS cannot be mounted by Windows 10
  • Same drive reformatted as exFAT by Windows 10 can be mounted on macOS for read/write

Conclusion: To use exFAT USB drive for portable storage, format the drive on Windows 10.

Tags: OSX, Windows