Minimizing Docker Pharo - Plugins

30 Nov 2019

In the quest^Wrather leisurely ambulation towards the smallest possible Docker image for Pharo for running headless, batch and server-side applications, one approach is to reduce the size of the Pharo VM, by removing irrelevant built-in and external plugins, also known as modules.

Here's what a pharo.cog.spur.minheadless VM built yesterday produces from STON toStringPretty: Smalltalk vm listBuiltinModules:

'[
	''SqueakFFIPrims'',
	''IA32ABI VMMaker.oscog-eem.2480 (i)'',
	''FilePlugin VMMaker.oscog-eem.2530 (i)'',
	''FileAttributesPlugin FileAttributesPlugin.oscog-eem.50 (i)'',
	''LargeIntegers v2.0 VMMaker.oscog-eem.2530 (i)'',
	''LocalePlugin VMMaker.oscog-eem.2495 (i)'',
	''MiscPrimitivePlugin VMMaker.oscog-eem.2480 (i)'',
	''SecurityPlugin VMMaker.oscog-eem.2480 (i)'',
	''SocketPlugin VMMaker.oscog-eem.2568 (i)'',
	''B2DPlugin VMMaker.oscog-eem.2536 (i)'',
	''BitBltPlugin VMMaker.oscog-nice.2587 (i)'',
	''FloatArrayPlugin VMMaker.oscog-eem.2480 (i)'',
	''FloatMathPlugin VMMaker.oscog-eem.2480 (i)'',
	''Matrix2x3Plugin VMMaker.oscog-eem.2480 (i)'',
	''DropPlugin VMMaker.oscog-eem.2480 (i)'',
	''ZipPlugin VMMaker.oscog-eem.2480 (i)'',
	''ADPCMCodecPlugin VMMaker.oscog-eem.2480 (i)'',
	''AsynchFilePlugin VMMaker.oscog-eem.2493 (i)'',
	''BMPReadWriterPlugin VMMaker.oscog-eem.2480 (i)'',
	''DSAPrims CryptographyPlugins-eem.14 (i)'',
	''FFTPlugin VMMaker.oscog-eem.2480 (i)'',
	''FileCopyPlugin VMMaker.oscog-eem.2493 (i)'',
	''JoystickTabletPlugin VMMaker.oscog-eem.2493 (i)'',
	''MIDIPlugin VMMaker.oscog-eem.2493 (i)'',
	''SerialPlugin VMMaker.oscog-eem.2493 (i)'',
	''SoundCodecPrims VMMaker.oscog-eem.2480 (i)'',
	''SoundGenerationPlugin VMMaker.oscog-eem.2480 (i)'',
	''StarSqueakPlugin VMMaker.oscog-eem.2480 (i)'',
	''Mpeg3Plugin VMMaker.oscog-eem.2495 (i)'',
	''VMProfileLinuxSupportPlugin VMMaker.oscog-eem.2480 (i)'',
	''UnixOSProcessPlugin VMConstruction-Plugins-OSProcessPlugin.oscog-dtl.66 (i)''
]'

And here's the directory listing of the VM as built:

~/src/opensmalltalk-vm/products/ph64mincogspurlinuxht% ls
libAioPlugin.so*             libPharoVMCore.a             libSqueakSSL.so*             libssh2.so@
libCroquetPlugin.so*         libRePlugin.so*              libSurfacePlugin.so*         libssh2.so.1@
libEventsHandlerPlugin.so*   libSDL2-2.0.so.0@            libcrypto.so.1.1*            libssh2.so.1.0.1*
libInternetConfigPlugin.so*  libSDL2-2.0.so.0.7.0*        libgit2.so@                  libssl.so@
libJPEGReadWriter2Plugin.so* libSDL2.so@                  libgit2.so.0.26.8*           libssl.so.1.1*
libJPEGReaderPlugin.so*      libSDL2DisplayPlugin.so*     libgit2.so.26@               pharo*

For server-side applications, a number of the plugins and shared libraries (certainly also libPharoVMCore.a) need not be part of the Docker image.

Tags: Alpine Linux, deployment, Docker

Minimizing Docker Pharo

11 Aug 2019

As tweeted: Docker image of Pharo VM, 7.0.3-based app Pharo image, changes/sources files, Ubuntu 16.04, 484MB. With Pharo VM Alpine Docker image, 299MB. Build app Pharo image from minimal, run without changes/sources, Alpine Pharo VM, 83MB!

And here's the Docker container resident set size upon switching from the 484MB Docker image to the Alpine-based Docker image:

Pharo Docker RSS

Norbert Hartl and I are collaborating on minimizing Dockerized Pharo. All are welcome to join.

Tags: Alpine Linux, deployment, Docker

PoC: Alpine Linux Minimal Stateless Pharo

20 Jul 2019

Alpine Linux is a security-oriented, lightweight distro based on musl-libc and BusyBox. Official Docker images of Alpine Linux are about 5MB each.

I've successfully built the pharo.cog.spur.minheadless OpenSmalltalk VM on Alpine Linux. Dockerizing the Pharo VM files plus a built-from-source libsqlite3.so (without Pharo image/changes/etc) produces a Docker image weighing in at 12.5MB.

% sudo docker images | egrep "samadhiweb.*alpine"
samadhiweb/pharo7vm                    alpine              299420ff0e03        21 minutes ago      12.5MB

Pharo provides minimal images that contain basic Pharo packages without the integrated GUI, useful for building server-side applications.

From the Pharo 7.0.3 minimal image, I've built a "stateless" image containing FFI, Fuel, and UDBC-SQLite. "Stateless" means the image can be executed by the Pharo VM without the changes and sources files. Here are the sizes of the base minimal image and my SQLite image:

% ls -l udbcsqlite.image Pharo7.0.3-0-metacello-64bit-0903ade.image 
-rw-rw-r-- 1 pierce pierce 13863032 Apr 12 22:56 Pharo7.0.3-0-metacello-64bit-0903ade.image
-rw-rw-r-- 3 pierce pierce 17140552 Jul 20 14:11 udbcsqlite.image

Below, I run the image statelessly on my Ubuntu laptop. Note that this uses the regular Pharo VM, not the Alpine Linux one.

% mkdir stateless
% cd stateless
% ln ../udbcsqlite.image
% chmod 444 udbcsqlite.image
% cat > runtests.sh <<EOF
#!/bin/sh
~/pkg/pharo7vm/gofaro -vm-display-none udbcsqlite.image test --junit-xml-output "UDBC-Tests-SQLite-Base"
EOF
% chmod a+x runtests.sh
% ls -l
total 16744
-rwxr-xr-x 1 pierce pierce      116 Jul 20 15:25 runtests.sh*
-r--r--r-- 3 pierce pierce 17140552 Jul 20 14:11 udbcsqlite.image
% 
%
% ./runtests.sh
Running tests in 1 Packages
71 run, 71 passes, 0 failures, 0 errors.
% ls -l 
total 16764
-rw-r--r-- 1 pierce pierce     6360 Jul 20 15:26 progress.log
-rwxr-xr-x 1 pierce pierce      116 Jul 20 15:25 runtests.sh*
-r--r--r-- 3 pierce pierce 17140552 Jul 20 14:11 udbcsqlite.image
-rw-r--r-- 1 pierce pierce    11072 Jul 20 15:26 UDBC-Tests-SQLite-Base-Test.xml

Dockerizing udbcsqlite.image together with the aforementioned Alpine Linux Pharo VM produces a Docker image that is 46.8 MB in size.

% sudo docker images | egrep "samadhiweb.*alpine"
samadhiweb/p7minsqlite                 alpine              3a57853099d0        44 minutes ago      46.8MB
samadhiweb/pharo7vm                    alpine              299420ff0e03        About an hour ago   12.5MB

Run the Docker image:

% sudo docker run --ulimit rtprio=2 samadhiweb/p7minsqlite:alpine
Running tests in 1 Packages
71 run, 71 passes, 0 failures, 0 errors.

For comparison and contrast, here are the sizes of the regular Pharo 7.0.3 image, changes and sources files:

% ls -l Pharo7.0*-0903ade.* 
-rw-rw-r-- 1 pierce pierce      190 Apr 12 22:57 Pharo7.0.3-0-64bit-0903ade.changes
-rw-rw-r-- 1 pierce pierce 52455648 Apr 12 22:57 Pharo7.0.3-0-64bit-0903ade.image
-rw-rw-r-- 2 pierce pierce 34333231 Apr 12 22:55 Pharo7.0-32bit-0903ade.sources
Tags: Alpine Linux, deployment, Docker

This blog now on HTTPS

09 Dec 2018

This blog is now on HTTPS.

Setup:

  • Caddy as web reverse proxy.
  • SmallCMS1, the blog engine, runs on Pharo 6.
  • Blog content is held in a Fossil repository with a running Fossil server to support content pushing.
  • Each component runs in a Docker container.

Caddy is an open source HTTP/2 web server. caddy-docker-proxy is a plugin for Caddy enabling Docker integration - when an appropriately configured Docker container or service is brought up, caddy-docker-proxy generates a Caddy site specification entry for it and reloads Caddy. With Caddy's built-in Let's Encrypt functionality, this allows the new container/service to run over HTTPS seamlessly.

Below is my docker-compose.yml for Caddy. I built Caddy with the caddy-docker-proxy plugin from source and named the resulting Docker image samadhiweb/caddy. The Docker network caddynet is the private network for Caddy and the services it is proxying. The Docker volume caddy-data is for persistence of data such as cryptographic keys and certificates.

version: '3.6'

services:

  caddy:
    image: samadhiweb/caddy
    command: -agree -docker-caddyfile-path=/pkg/caddy/caddyfile -log=/var/log/caddy/caddy.log
    ports: 
      - "80:80"
      - "443:443"
    networks: 
      - caddynet
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /pkg/caddy
        target: /pkg/caddy
      - type: volume
        source: caddy-data
        target: /root/.caddy
      - type: bind
        source: /var/log/caddy
        target: /var/log/caddy
    restart: unless-stopped

networks:
  caddynet:
    name: caddynet
    external: true

volumes:
  caddy-data:
    name: caddy-data
    external: true

Here's the docker-compose.yml snippet for the blog engine:

services:
  scms1app:
    image: samadhiweb/scms1app
    ports: 
      - "8081:8081"
    networks:
      - caddynet
    volumes:
      - type: bind
        source: /pkg/smallcms1/config.json
        target: /etc/smallcms1/config.json
      - type: volume
        source: smdw-content
        target: /pkg/cms
    labels:
      - "caddy.address=www.samadhiweb.com"
      - "caddy.targetport=8081"
      - "caddy.targetprotocol=http"
      - "caddy.proxy.header_upstream_1=Host www.samadhiweb.com"
      - "caddy.proxy.header_upstream_2=X-Real-IP {remote}"
      - "caddy.proxy.header_upstream_3=X-Forwarded-For {remote}"
      - "caddy.tls=email-address@samadhiweb.com"
      - "caddy.log=/var/log/caddy/www.samadhiweb.com.access.log"
    ulimits:
      rtprio:
        soft: 2
        hard: 2
    restart: unless-stopped

networks:
  caddynet:
    name: caddynet
    external: true

volumes:
  smdw-content:
    name: smdw-content
    external: true

Of interest are the caddy.* labels from which caddy-docker-proxy generates the following in-memory Caddy site entry:

www.samadhiweb.com {
  log /var/log/caddy/www.samadhiweb.com.access.log
  proxy / http://<private-docker-ip>:8081 {
    header_upstream Host www.samadhiweb.com
    header_upstream X-Real-IP {remote}
    header_upstream X-Forwarded-For {remote}
  }
  tls email-address@samadhiweb.com
}

Also note the ulimits section, which sets the suggested limits for the Pharo VM heartbeat thread. These limits must be set in the docker-compose file or on the docker command line - copying a prepared file into /etc/security/limits.d/pharo.conf does not work when run in a Docker container.

    ulimits:
      rtprio:
        soft: 2
        hard: 2
Tags: deployment, Docker

Docker and Pharo

09 Sep 2018

Recently there were discussions and blog posts on Docker for Pharo and Gemstone/S. This is my report after spending an afternoon on the subject.

First, some links:

This blog is implemented in Pharo and is the natural choice for my Docker example application. I already have a Smalltalk snippet to load this blog's code and its dependencies into a pristine Pharo image, so I'll be using that. Also, as a matter of course, I build the Pharo VM from source, and my VM installation also contains self-built shared libraries like libsqlite.so and libshacrypt.so.

Outside of Docker, prepare a custom Pharo image:

% cp ../Pharo64-60543.image scms1.image
% cp ../Pharo64-60543.image scms1.image
% ~/pkg/pharo6vm64/gofaro scms1.image st loadSCMS1.st

gofaro is a simple shell script which purpose is to make sure the Pharo VM loads my custom shared libraries, co-located with the standard VM files, at run time:

#!/bin/sh
PHAROVMPATH=$(dirname `readlink -f "$0"`)
LD_LIBRARY_PATH="$PHAROVMPATH" exec "$PHAROVMPATH/pharo" $@

loadSCMS1.st looks like this:

"Load dependencies and then the blog code."
Gofer it ...
Gofer it ...
Metacello new ...
Metacello new ...

"Save the image for injection into Docker."
SmalltalkImage current snapshot: true andQuit: true

Before describing my Dockerfile, here are my conventions for inside the Docker container:

  • VM goes into /pkg/vm.
  • Application artifacts including the image and changes files go into /pkg/image.
  • For this blog application, the blog's content is in /pkg/cms.

Starting with Ubuntu 18.04, install libfreetype6. The other lines are copied from Torsten's tutorial.

FROM ubuntu:18.04
LABEL maintainer="Pierce Ng"
RUN apt-get update \
  && apt-get -y install libfreetype6 \
  && apt-get -y upgrade \
  && rm -rf /var/lib/apt/lists/* \
  && true

Next, install the Pharo VM.

RUN mkdir -p /pkg/vm
COPY pharo6vm64/ /pkg/vm
COPY pharolimits.conf /etc/security/limits.d/pharo.conf

Now copy over the prepared Pharo image.

RUN mkdir -p /pkg/image
WORKDIR /pkg/image
COPY PharoV60.sources PharoV60.sources
COPY scms1.image scms1.image
COPY scms1.changes scms1.changes
COPY runSCMS1.st runSCMS1.st

Finally, set the Docker container running. Here we create a UID/GID pair to run the application. Said UID owns the mutable Pharo files in /pkg/image and also the /pkg/image directory itself, in case the application needs to create other files such as SQLite databases.

RUN groupadd -g 1099 pharoapp && useradd -r -u 1099 -g pharoapp pharoapp
RUN chown -R pharoapp:pharoapp /pkg/image
RUN chown root:root /pkg/image/PharoV60.sources
RUN chown root:root /pkg/image/runSCMS1.st
EXPOSE 8081
USER pharoapp:pharoapp
CMD /pkg/vm/gofaro -vm-display-null -vm-sound-null scms1.image --no-quit st runSCMS1.st

runSCMS1.st runs the blog application. In my current non-Dockerized installation, the runSCMS1.st-equivalent snippet is in a workspace; for Docker, to become DevOps/agile/CI/CD buzzwords-compliant, this snippet is run from the command line. This is one Dockerization adaptation I had to make to my application.

Now we build the Docker image.

% sudo docker build -t samadhiweb/scms1:monolithic .
Sending build context to Docker daemon    299MB
Step 1/19 : FROM ubuntu:18.04
 ---> cd6d8154f1e1
Step 2/19 : LABEL maintainer="Pierce Ng"
 ---> Using cache
 ---> 1defb3ac00a8
Step 3/19 : RUN apt-get update   && apt-get -y install libfreetype6   && apt-get -y upgrade   && rm -rf /var/lib/apt/lists/*   && true
 ---> Running in b4e328138b50
<bunch of apt-get output>
Removing intermediate container b4e328138b50
 ---> 79e9d8ed7959
Step 4/19 : RUN mkdir -p /pkg/vm
 ---> Running in efb2b9b717fe
Removing intermediate container efb2b9b717fe
 ---> 0526cbc4c483
Step 5/19 : COPY pharo6vm64/ /pkg/vm
 ---> 2d751994c68c
Step 6/19 : COPY pharolimits.conf /etc/security/limits.d/pharo.conf
 ---> f442f475c568
Step 7/19 : RUN mkdir -p /pkg/image
 ---> Running in 143ebd54f243
Removing intermediate container 143ebd54f243
 ---> 6d1b99d30050
Step 8/19 : WORKDIR /pkg/image
 ---> Running in 45c76d8c08c0
Removing intermediate container 45c76d8c08c0
 ---> 57247408801b
Step 9/19 : COPY PharoV60.sources PharoV60.sources
 ---> 8802acc416f0
Step 10/19 : COPY scms1.image scms1.image
 ---> 3e2d62be5d00
Step 11/19 : COPY scms1.changes scms1.changes
 ---> dcbec7ebdda9
Step 12/19 : COPY runSCMS1.st runSCMS1.st
 ---> 72fa4efb33ff
Step 13/19 : RUN groupadd -g 1099 pharoapp && useradd -r -u 1099 -g pharoapp pharoapp
 ---> Running in e0af716c8db2
Removing intermediate container e0af716c8db2
 ---> 0a42beed8065
Step 14/19 : RUN chown -R pharoapp:pharoapp /pkg/image
 ---> Running in 2da21fefa399
Removing intermediate container 2da21fefa399
 ---> 0d808f48ae32
Step 15/19 : RUN chown root:root /pkg/image/PharoV60.sources
 ---> Running in 4ca0c6eb8301
Removing intermediate container 4ca0c6eb8301
 ---> 1426236b509c
Step 16/19 : RUN chown root:root /pkg/image/runSCMS1.st
 ---> Running in a942ecb8a155
Removing intermediate container a942ecb8a155
 ---> 1213e1647076
Step 17/19 : EXPOSE 8081
 ---> Running in 3b74e55b6394
Removing intermediate container 3b74e55b6394
 ---> a04593571d13
Step 18/19 : USER pharoapp:pharoapp
 ---> Running in 77ecde5a7ca7
Removing intermediate container 77ecde5a7ca7
 ---> 975b614d3a9f
Step 19/19 : CMD /pkg/vm/gofaro -vm-display-null -vm-sound-null scms1.image --no-quit st runSCMS1.st
 ---> Running in 2c6e7645da3d
Removing intermediate container 2c6e7645da3d
 ---> 65b4ca6cc5c5
Successfully built 65b4ca6cc5c5
Successfully tagged samadhiweb/scms1:monolithic
% sudo docker image ls
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
samadhiweb/scms1        monolithic          65b4ca6cc5c5        2 minutes ago       402MB
...
% 

The Docker image has been created, but it is not ready to run yet, because the web content is not in the image. I'll put the content in a Docker volume. Below, the first -v mounts my host's content directory into /tmp/webcontent in the container; the second -v mounts the volume smdw-content into /pkg/cms in the container; I'm running the busybox image to get a shell prompt; and within the container I copy the web content from the source to the destination.

% sudo docker volume create smdw-content
% sudo docker run --rm -it \
  -v ~/work/webcms/samadhiweb:/tmp/webcontent \
  -v smdw-content:/pkg/cms \
  busybox sh
/ # cp -p -r /tmp/webcontent/* /pkg/cms/
/ # ^D
% 

Finally, run the Docker image, taking care to mount the volume smdw-content, now with this blog's content:

% sudo docker run --rm -d -p 8081:8081 \
  -v smdw-content:/pkg/cms \
  --name samadhiweb samadhiweb/scms1:monolithic
bfcc80b32f35b3979c5c8c1b28bd3464f79ebdae91f51d9422334b209678ab5c
% sudo docker ps
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS              PORTS                    NAMES
bfcc80b32f35        samadhiweb/scms1:monolithic   "/bin/sh -c '/pkg/vm."   4 seconds ago       Up 3 seconds        0.0.0.0:8081->8081/tcp   samadhiweb

Verified with a web browser. This works on my computer. :-)

Tags: deployment, Docker

LibreSSL and Pharo/Squeak SSL plugin

22 Oct 2015

According to its documentation, on Unix, the Pharo VM's SSL plugin, libSqueakSSL.so, links into OpenSSL libraries dynamically. On my 64bit Ubuntu Trusty machine, OpenSSL is provided by the libssl1.0.0:i386 package.

$ ldd libSqueakSSL.so
    linux-gate.so.1 =>  (0xf77a9000)
    libssl.so.1.0.0 => /lib/i386-linux-gnu/libssl.so.1.0.0 (0xf7727000)
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7579000)
    libcrypto.so.1.0.0 -> /lib/i386-linux-gnu/libcrypto.so.1.0.0 (0xf73cb000)
    /lib/ld-linux.so.2 (0xf77a9000)
    libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf73c6000)

(By the way, the SSH2 plugin libssh2.so.1.0.1 requires libcrypto too.)

According to packages.ubuntu.com, Trusty's libssl1.0.0 is built from openssl_1.0.1f.orig.tar.gz plus successive upstream patches.

From the OpenBSD developers, LibreSSL is "a version of the TLS/crypto stack forked from OpenSSL in 2014, with goals of modernizing the codebase, improving security, and applying best practice development processes." LibreSSL also comes with libtls, "a new TLS library, designed to make it easier to write foolproof applications".

Let's see how we go about linking libSqueakSSL.so with LibreSSL.

First, download and unpack LibreSSL. Modify the configure script at lines 2287 and 2289 so that LIBCRYPTO_VERSION and LIBSSL_VERSION both say 1:0:0 instead of 35:0:0. Then build LibreSSL:

$ CFLAGS=-m32 LDFLAGS=-m32 ./configure --disable-asm
$ make

I'm building on a 64bit OS, hence "-m32". Without "--disable-asm", the build fails. To get the assembler version, which is recommended for serious usage, either set up a 32bit build environment or muck around with autoconf/configure. I suspect the former is easier. :-)

The output files are $SRC/crypto/.libs/libcrypto.so.1.0.0 and $SRC/ssl/.libs/libssl.so.1.0.0. The shared object files have the "1.0.0" suffix because I modified configure above. Alternatively, I could've played around with autoconf, or built the shared objects with the "35.0.0" suffix and sym/hard-link them for the "1.0.0" versions. TIMTOWTDI.

Next, remove the OpenSSL package:

$ sudo apt-get remove libssl1.0.0:i386
$ ldd libSqueakSSL.so
    linux-gate.so.1 =>  (0xf7718000)
    libssl.so.1.0.0 => not found
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7540000)
    /lib/ld-linux.so.2 (0xf7718000)

Finally, put the LibreSSL shared object files into the right place. Where this right place is depends on your environment. TIMTOWTDI. I choose to put them in the Pharo VM directory with its other plugins, and arrange to start Pharo with LD_LIBRARY_PATH set appropriately. Going by the output of ldd again, the following is required:

$ ln libcrypto.so.1.0.0 libcrypto.so.1

After which:

$ ldd libSqueakSSL.so
    linux-gate.so.1 =>  (0xf7709000)
    libssl.so.1.0.0 => /pkg/pharovm/libssl.so.1.0.0 (0xf7696000)
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf74c7000)
    libcrypto.so.1 -> /pkg/pharovm/libcrypto.so.1 (0xf72b8000)
    /lib/ld-linux.so.2 (0xf770a000)

Launch the Pharo 4.0 image and run the Zodiac tests. All tests should pass. Well, except testGetPharoVersion, which looks for a file that apparently no longer exists.

Incidentally, Squeak 5.0-All-in-One's SSL plugin appears to have linked its crypto/SSL libraries in statically, so the only way to upgrade is to build a new plugin.

$ ldd SqueakSSL
    linux-gate.so.1 =>  (0xf7736000)
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7310000)
    /lib/ld-linux.so.2 (0xf7737000)
Tags: cryptography, deployment, Linux, security

Pharo Smalltalk on Ubuntu 14.04 64bit

20 Apr 2014

I run Debian 6 64bit on one of my servers. To run the 32bit Pharo/Cog VM, 32bit libraries are needed and are installed thusly:

# apt-get install ia32-libs

Trying out the newly released Ubuntu 14.04, I learn that ia32-libs is considered a hack and was deprecated in 2013 versions of Ubuntu. I guess the idea is to encourage people to know what they require more precisely.

To get the current Pharo-VM-linux-stable.zip to work on Ubuntu 14.04 64bit, install the following packages:

# apt-get lib32gcc1 
# apt-get libssl0.9.8:i386

On libssl, that is, OpenSSL, Ubuntu also ships 1.0.1f-1ubuntu2, which patches the Heartbleed bug, so you may install that instead.

Tags: CogVM, deployment, Linux

Pharo Headless Interactive RFB

04 Nov 2012

In deployment war stories, I wrote that Pharo's graphical environment is managed by MorphicUIManager, but when invoked -nodisplay, Pharo's graphical environment is managed by NonInteractiveUIManager.

I prefer running my servers without X as far as possible, so I want to have RFBServer running within the Pharo image, to be able to VNC into it.

As it turns out, modifying one method, MorphicUIManager>>onSnapshot:, gets me what I want. Here's what the method looks like in Pharo 1.4, timestamped CamilloBruni 2/13/2012 23:22, (minus comments because I don't like the way the Smalltalk syntax highlighting JS library I'm using is rendering them):

onSnapshot: resuming

  resuming ifTrue: [
    Smalltalk isInteractive ifFalse: [
      ^ self nonInteractiveManager onSnapshot: resuming ].
    Smalltalk isHeadless ifTrue: [
      ^ self headlessManager onSnapshot: resuming ]].

  SystemWindow wakeUpTopWindowUponStartup

I just needed to add tests for RFBServer, so now the method becomes like this:

onSnapshot: resuming
   	
  resuming ifTrue: [
    Smalltalk isInteractive ifFalse: [
      RFBServer server isRunning ifFalse: [
        ^ self nonInteractiveManager onSnapshot: resuming ]].
    Smalltalk isHeadless ifTrue: [
      RFBServer server isRunning ifFalse: [	
        ^ self headlessManager onSnapshot: resuming ]]].
   
  SystemWindow wakeUpTopWindowUponStartup

And now VNC into the Pharo image works, yay!

Tags: deployment, headless, RFB, VNC

Deployment War Stories

28 Oct 2012

I used to run Squeak and subsequently Pharo on FreeBSD. I had the Smalltalk RFB server running within the image, and connected to the image via COTVNC (Chicken of the VNC), an OSX client. I ran Squeak/Pharo in a FreeBSD jail, and doing it this way kept the jail minimalist and tidy. No X stuff was installed.

Then I ran into some difficulty with FFI on CogVM on FreeBSD. (That's for another post.) And I was running FreeBSD 7, which EOLed. So I switched to Linux. Chose Debian 6 after trying out several distros. Similar setup as on FreeBSD: Minimalist LXC container, CogVM, RFB server in the same Pharo image that ran on FreeBSD.

Interestingly, neither COTVNC nor several other VNC clients (both within X on Linux and from OSX, including Java and native ones) worked in this setup: As far as I determined, the VNC connections were all failing because of incompatible encodings, and I tested quite a few combinations.

I wasn't interested in debugging too much, so I did the next simplest thing: Installed an Xvnc environment. So, instead of running an RFB server within Pharo, I had Pharo running within an X environment that was also a VNC server. And then all the VNC clients I was testing with worked. Not as tidy as I wanted, because now the LXC container had a whole bunch of X cruft, but hey, it was working.

But I really prefer FreeBSD. So after getting my production Linux server up and running, I went to investigate my FreeBSD CogVM FFI issue. And it turned out to be a process error on my part. I think. But that's for another post.

A note on my (fairly typical) setup: My development machine is an MBP. My development and pre-production FreeBSD and Linux servers all exist as VirtualBox or VMWare virtual machines on my MBP. The development servers have X, while the pre-production servers are meant to provide the canonical configuration for the production servers, so I run them without X, as much as possible. On FreeBSD, I have never had to run X in production. On Linux, I have the aforementioned Xvnc setup.

Anyhow, CogVM FFI FreeBSD was working again, in my development server running X. Next I tested it in my X-less pre-production server. When I invoked CogVM with "-headless", it said "cannot locate vm-display-X11" or words to that effect, even though said dynamic library was right there together with vm-display-null. I reckon that vm-display-X11 failed to locate an X environment, hence failed to load, hence the error message. Meaning, instead of -headless, I had to run -nodisplay.

So, CogVM ran -nodisplay, and COTVNC connected to it fine. Opened a workspace, typed some FFI-using code, selected the text with the mouse, pressed ESC. What's this? CogVM or Pharo crashed! PharoDebugLog recorded "SubclassResponsibility: NonInteractiveUIManager had the subclass responsibility to implement #newMenuIn:for:".

Hmmm. This didn't happen in Pharo 1.0 and 1.1, IIRC. Seems Pharo's random refactorings (heh, just kidding) changed how -nodisplay was managed between those versions and 1.4, the version I was testing.

After browsing a bit, looks like Pharo's graphical environment is managed by MorphicUIManager, but when invoked -nodisplay, even with a running RFB server, Pharo's graphical environment is managed by NonInteractiveUIManager.

One possibility to get this working is to install a minimal X environment, to run CogVM+Pharo -headless (with MorphicUIManager) instead of -nodisplay (with NonInteractiveUIManager).

But that isn't the simplest thing that could possibly work. What is? Copy & paste, of course. Specifically, I copied/pasted MorphicUIManager>>newMenuIn:for: into NonInteractiveUIManager. Ok, now pressing ESC brings up the context menu, "Do it" does, and output in the Transcript shows the expected FFI-related output.

Next I run my application's startup code in a workspace, one step of which is sending the "explore" message to the application's main instance, thus giving me a handle to said instance should I need to manipulate it, which is kinda the point of having VNC access into the image in the first place.

What's this? Crashed again! PharoDebugLog informs me "MessageNotUnderstood: NonInteractiveUIManager>>explorer:for:withLabel:" and "An attempt to use interactive tools detected, while in non-interactive mode".

Not so simple then...

(To be continued.)

Tags: deployment, FreeBSD, Linux, RFB, VNC