Nvidia X11 hardware acceleration on Ubuntu 18.04

19 February 2019

After reinstalling Windows 7, I set my laptop up to dual boot Xubuntu 18.04. My laptop has an integrated Intel i915 graphics chip and also an Nvidia GeForce GT 330M GPU. This post describes how I set up X11 to use the GPU.

Start by finding out which drivers are in use and available:

% ubuntu-drivers devices
... currently running nouveau
... recommends nvidia-340

Install the recommended driver and reboot:

% sudo apt-get install nvidia-340
...
% sudo shutdown -r now

After the reboot, VT-7 showed a black screen. VTs 1-6 were still available and the machine was otherwise functional. X.org.0.log recorded that the Nvidia driver had set (the graphics) mode to NULL.

To fix:

% sudo apt-get install nvidia-prime
% sudo prime-select nvidia
% sudo shutdown -r now

The X environment came back. To validate:

% inxi -G
Graphics:  Card-1: Intel Core Processor Integrated Graphics Controller
           Card-2: NVIDIA GT216M [GeForce GT 330M]
           Display Server: x11 (X.Org 1.19.6 ) drivers: modesetting,nvidia (unloaded: fbdev,vesa,nouveau)
           Resolution: 1920x1080@59.94hz
           OpenGL: renderer: GeForce GT 330M/PCIe/SSE2 version: 3.3.0 NVIDIA 340.107

There's a tool to switch between the Nvdia and Intel chips:

Nvidia Profile Switching

Also:

% glxgears -info
...
...
13122 frames in 5.0 seconds = 2624.275 FPS
13039 frames in 5.0 seconds = 2607.778 FPS
13916 frames in 5.0 seconds = 2783.134 FPS
11971 frames in 5.0 seconds = 2394.196 FPS

For comparison and contrast, an even older GPU-less laptop managed about ~60 FPS running glxgears.

As for Pharo, the OSWindow external operating system window examples, such as SDL2AthensDrawingExample, now only run in Nvidia mode and crash Pharo instantly in Intel mode:

The program 'pharo' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadValue (integer parameter out of range for operation)'.
  (Details: serial 100 error_code 2 request_code 154 minor_code 3)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the --sync command line
   option to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)

Building OpenSmalltalk VM for Pharo on Win 7

3 February 2019

I had the occasion to reinstall Windows 7 on my laptop and took the chance to set it up with the latest development tools. The objective was to build OpenSmalltalk for Pharo and Dolphin Smalltalk on the machine.

Install Visual Studio 2017 Community Edition

This is primarily for building Dolphin Smalltalk. The command line tools come in handy when building SQLite.

I chose to install most VS2017 options and it weighed in at a whopping 50GB of disk space.

Install Cygwin

I installed the 64-bit edition. The following packages are required to build 32-bit OpenSmalltalk.

  • cmake
  • gcc-core
  • gcc-g++
  • libglib2.0-devel
  • libglib2.0
  • make
  • mingw-i686-gcc-core
  • mingw-i686-gcc-g++
  • mingw-i686-headers
  • mingw-i686-runtime
  • patch
  • perl-base
  • wget
  • zip

iconv.dll

Jumping ahead a bit, as part of the build process, the OpenSmalltalk build system attempts to copy /usr/i686-w64-mingw32/sys-root/mingw/bin/iconv.dll. On my Cygwin installation, there is no such file, but there does exist /usr/bin/cygiconv-2.dll. To make things simple, I copied /usr/bin/cygiconv-2.dll over to where OpenSmalltalk expects to find it. Looking around the web, I believe cygiconv-2.dll is the correct file.

Build OpenSmalltalk

Start a Cygwin terminal. Clone the OpenSmalltalk VM repo and build the VM.

$ mkdir -p /cygdrive/c/Users/Pierce/source/repos/opensmalltalk
$ cd /cygdrive/c/Users/Pierce/source/repos/opensmalltalk
$ git clone https://github.com/opensmalltalk/opensmalltalk-vm.git
...
Checking out files: 100% (10541/10541), done.
$ cd opensmalltalk-vm/build.win32x86/pharo.cog.spur
$ ../../scripts/updateSCCSVersions
$ ./mvm -f
<humongous amount of output, much fan noise and heat dissipation from laptop>
$ ls build/vm
btext.o           Pharo.def           sqNamedPrims.o          sqWin32PluginSupport.o
cogit.o           Pharo.exe           sqPath.o            sqWin32Prefs.o
etext.o           Pharo.exe.manifest          sqTextEncoding.o        sqWin32Service.o
FileAttributesPlugin.dll  Pharo.exp           sqTicker.o          sqWin32SpurAlloc.o
FT2Plugin.dll         Pharo.lib           SqueakSSL.dll       sqWin32Stubs.o
gcc3x-cointerp.o      Pharo.map           sqVirtualMachine.o      sqWin32Threads.o
iconv.dll         Pharo.res           sqWin32Alloc.o          sqWin32Time.o
libcairo-2.dll        PharoConsole.exe        sqWin32Backtrace.o      sqWin32Utils.o
libeay32.dll          PharoConsole.exe.manifest   sqWin32DirectInput.o    sqWin32VMProfile.o
libfreetype.dll       PharoConsole.map        sqWin32Directory.o      sqWin32Window.o
libgcc_s_sjlj-1.dll   PharoConsoleUnstripped.exe  sqWin32DnsInfo.o        ssleay32.dll
libgit2.dll       PharoUnstripped.exe         sqWin32Exports.o        SurfacePlugin.dll
libpixman-1-0.dll     resource.o              sqWin32ExternalPrims.o  version.o
libpng16-16.dll       SDL2.dll            sqWin32GUID.o       zlib1.dll
libssh2-1.dll         sqExternalSemaphores.o      sqWin32Heartbeat.o
libwinpthread-1.dll   sqHeapMap.o             sqWin32Main.o

Now open a cmd.exe window and copy the VM files out.

C:\> mkdir c:\pkg\pharo6vm32
C:\> copy c:\Users\Pierce\source\repos\opensmalltalk\opensmalltalk-vm\build.win32x86\pharo.cog.spur\build\vm\*.dll c:\pkg\pharo6vm32
C:\> copy c:\Users\Pierce\source\repos\opensmalltalk\opensmalltalk-vm\build.win32x86\pharo.cog.spur\build\vm\*.exe c:\pkg\pharo6vm32
C:\>dir /d c:\pkg\pharo6vm32
 Volume in drive C has no label.
 Volume Serial Number is FCE0-E161

 Directory of c:\pkg\pharo6vm32

[.]                          libssh2-1.dll
[..]                         libwinpthread-1.dll
FileAttributesPlugin.dll     Pharo.exe
FT2Plugin.dll                PharoConsole.exe
iconv.dll                    PharoConsoleUnstripped.exe
libcairo-2.dll               PharoUnstripped.exe
libeay32.dll                 SDL2.dll
libfreetype.dll              SqueakSSL.dll
libgcc_s_sjlj-1.dll          ssleay32.dll
libgit2.dll                  SurfacePlugin.dll
libpixman-1-0.dll            zlib1.dll
libpng16-16.dll
              21 File(s)     47,771,093 bytes
               2 Dir(s)  29,938,479,104 bytes free

Build SQLite DLL

Download and unpack the SQLite source amalgamation. The how to compile SQLite page states to use the following MSVC command:

cl sqlite3.c -link -dll -out:sqlite3.dll

While this produces a DLL, said DLL doesn't actually export any symbol and is thus useless for FFI. This blog post by Mario Guggenberger provides the magic incantation:

cl sqlite3.c -DSQLITE_API=__declspec(dllexport) -link -dll -out:sqlite3.dll

To get to the 32-bit MSVC command line tools, click through the following: Windows Menu, All Programs, Visual Studio 2017, Visual Studio Tools, VC, x86 Native Tools Command Prompt. This launches a cmd.exe shell with the appropriate tools set up.

Build sqlite3.dll and copy it to c:\pkg\pharo6vm32.

Test with Pharo 7

Download the Pharo 7 32-bit image zip. Unpack and make a copy - I named the image file wintesting. Double click on c:\pkg\pharo6vm32\Pharo.exe and choose wintesting.image to start.

In Pharo, open Catalog Browser. Select and install GlorpSQLite. It fails. Run below Metacello snippet, which is what is executed when loading GlorpSQLite from Catalog Browser.

Metacello new
  baseline: 'GlorpSQLite';
  repository: 'github://PierceNg/glorp-sqlite3:pharo7';
  load

Fails with message "The handle is in the wrong state for the requested operation".

IceGenericError

There is a related Iceberg issue on GH. The suggestion is to use a newer version of libgit2.

Build newer libgit2

The current version of libgit2 used is 0.25.1, according to opensmalltalk-vm/third-party/libgit2.spec.

libgit2_spec_download_url:=https://github.com/libgit2/libgit2/archive/v0.25.1.tar.gz
libgit2_spec_archive_name:=libgit2-v0.25.1.tar.gz
libgit2_spec_unpack_dir_name:=libgit2-0.25.1
libgit2_spec_product_name_macOS:=libgit2.0.25.1.dylib
libgit2_spec_product_name_linux:=libgit2.so.0.25.1
libgit2_spec_product_name_windows:=libgit2.dll
libgit2_spec_symlinks_macOS:=libgit2*.dylib
libgit2_spec_symlinks_linux:=libgit2.so*
#libgit2_spec_download_url:=https://github.com/libgit2/libgit2/archive/v0.23.0.tar.gz
#libgit2_spec_archive_name:=libgit2-v0.23.0.tar.gz
#libgit2_spec_unpack_dir_name:=libgit2-0.23.0
#libgit2_spec_product_name_macOS:=libgit2.0.23.0.dylib
#libgit2_spec_product_name_linux:=libgit2.so.0.23.0
#libgit2_spec_product_name_windows:=libgit2.dll
#libgit2_spec_symlinks_macOS:=libgit2*.dylib
#libgit2_spec_symlinks_linux:=libgit2.so*

According to libgit2 GH repo, the newest versions are 0.27.8 and 0.26.8.

I first tried 0.27.8 but building failed. I then tried 0.26.8 and this time it built successfully. This is what libgit2.spec looks like now:

libgit2_spec_download_url:=https://github.com/libgit2/libgit2/archive/v0.26.8.tar.gz
libgit2_spec_archive_name:=libgit2-v0.26.8.tar.gz
libgit2_spec_unpack_dir_name:=libgit2-0.26.8
libgit2_spec_product_name_macOS:=libgit2.0.26.8.dylib
libgit2_spec_product_name_linux:=libgit2.so.0.26.8
libgit2_spec_product_name_windows:=libgit2.dll
libgit2_spec_symlinks_macOS:=libgit2*.dylib
libgit2_spec_symlinks_linux:=libgit2.so*

To build the newer version of libgit2 and the rest of the VM, re-run ./mvm -f in build.win32x86\pharo.cog.spur. When done, copy the .exe and .dll files into c:\pkg\pharo6vm32.

2nd Test

Start Pharo again and repeat steps described above in section Test with Pharo 7. This time Catalog Browser loads GlorpSQLite successfully.

GlorpSQLite on Pharo 7 on Windows 7

Open Test Runner and run Glorp tests. All tests pass.

GlorpSQLite3 on Pharo 7 on Windows 7

Updated README for PasswordCrypt

26 January 2019

I gave myself a 1Q2019 target to write proper READMEs for my Github repos. First up is PasswordCrypt. I reproduce the README here.

PasswordCrypt

PasswordCrypt is a library for Pharo Smalltalk to handle passwords salted and hashed by SHA-256/SHA-512. Its primary components are PCPasswordCrypt, PCAuthenticator and PCBasicAuthenticator.

PCPasswordCrypt

At its core, PCPasswordCrypt provides the following class-side messages:

  • sha512crypt:withSalt:
  • sha256crypt:withSalt:

An example:

PCPasswordCrypt sha256crypt: 'secret' withSalt: 'andPepperToo'
"'$5$andPepperToo$5p0MWgRMT6l6EA6dYDlFhuQKi.tfCXNd35T99HxbsTD'"

The result is a string in modular crypt format (MFC). $5 on the left of the string indicates that the hashing algorithm is SHA-256. For SHA-512, the indicator is $6.

On the instance side, PCPasswordCrypt generates the salt randomly if one is not supplied:

PCPasswordCrypt new sha256crypt: 'secret'
"'$5$5bUAI5i2$iIdIXcQGhZfNF0HQFG592Ut1I6UtuO/smBPJkKBrRzC'"

PCAuthenticator

PCAuthenticator builds on PCPasswordCrypt to provide username/password management. PCAuthenticator operates as a singleton object to persist its data in the Pharo image across restarts.

Example usage:

| appName auth newUser userToValidate |

appName := 'myApp'.

"Initialize the authenticator for my application."
auth := PCAuthenticator uniqueInstance.
auth initializeDatabaseFor: appName.

"Add a user."
newUser := PCUserCredential
  appname: appName;
  username: 'testuser';
  password: 'secret';
  yourself.
auth insertUserCredential: newUser.

"Create another user object and validate its password."
userToValidate := PCUserCredential
  appname: appName;
  username: 'testuser';
  password: 'secret';
  yourself.
auth validateUserCredential: userToValidate
"If the passwords match, userToValidate is returned; otherwise, nil is returned."

PCAuthenticatorUI is a simple Spec-based user interface to upsert new usernames/passwords into PCAuthenticator. I wrote it because I simply abhor code snippets containing clear-text passwords, except for demonstration as above. To run PCAuthenticatorUI:

PCAuthenticatorUI new openWithSpec

PCBasicAuthenticator

PCBasicAuthenticator subclasses ZnBasicAuthenticator, the HTTP basic authentication handler in ZincHTTPComponents. It uses PCAuthenticator so that

  • usernames and passwords are persistent,
  • it is feasible to use PCAuthenticatorUI for a small number of users, such as during development and testing.

Installation

To install the Pharo code:

Metacello new
  baseline: 'PasswordCrypt';
  repository: 'github://PierceNg/PasswordCrypt/src-st';
  load.

PCPasswordCrypt is an FFI to the C library libshacrypt, built from the source files shacrypt256.c and shacrypt512.c in the directory src-c. To build the C library:

% cd src-c
% make

The generated shared library is libshacrypt.so on Linux and libshacrypt.dylib on OSX/macOS. It must be placed where the Pharo VM can find it at run time. My practice is to place the shared library file together with the Pharo VM's plugins. On macOS, suppose Pharo is installed in /Users/pierce/Pharo.app, then libshacrypt.dylib goes into /Users/pierce/Pharo.app/Contents/MacOS/Plugins/.

Possible Future Work

  • Store/retrieve usernames/passwords in htpasswd files.

References

Licenses

  • MIT license for PasswordCrypt
  • sha256crypt.c and sha512crypt.c are public domain

OSSubprocess 64-bit macOS Mojave

15 January 2019

OSSubprocess works on macOS Mojave with 64-bit Pharo.

Load it the usual way:

Metacello new
  baseline: 'OSSubprocess';
  repository: 'github://pharo-contributions/OSSubprocess:master/repository';
  load. 

Run the tests. If Pharo was started like how applications are usually started on the Mac, by double-clicking the Pharo.app icon, Test Runner will report 40+ test errors.

Run Pharo the Unix way on the command line: ~/MyApps/Pharo.app/Contents/MacOS/Pharo mypharo7.image. Now all 114 tests pass. All 7 stress tests pass too. Tested with Pharo 7.0 pre-release image.