GOODS OODB on Unix Domain Sockets

28 Jul 2013

GOODS "is an object oriented fully distrbuted database management system using an active client model." It is also described as a "language-neutral object database" with client interfaces for C++, Java and Perl. GOODS is written by Konstantin Knizhnik.

Avi Bryant developed a Squeak client for GOODS that allows transparent storage of Smalltalk objects. The client is now maintained for Pharo, Squeak and VisualWorks by David Shaffer and is hosted on SqueakSource3.

GOODS works over TCP and Unix domain sockets. However, the GOODS documentation doesn't actually describe how to configure for the latter. There is, however, a hint in the main configuration file "goodsrv.cfg":

# Enable or disable acceptance of remote connections by GOODS server. 
# Disabling remote connections avoid any problem with firewall. In this case
# GOODS server is able to handle only local connections (Unix socket, Win32 local socket, process socket) 
server.remote.connections=1 # 1 or 0

In addition to the main configuration file, there is database-specific configuration file, which I've named "test.cfg", that looks like this:

1
0:localhost:60060

This matches the configuration format:

<number of storages = N>
<storage identifier 0>: <hostname>:<port>

Meaning, for test.cfg, I'm specifying one storage server, and it listens on TCP port 60060 on localhost.

But what about the Unix domain socket path? To find out, I set the server.remote.connections parameter to 0 and try it out:

% goodsrv test 0 -
Read the whole database size to page pool
16:52.59 28-JUL-2013: Checkpoint 37 finished
GOODS server started...
server is up...
>

What happened? Looking around, it is found that GOODS has created a Unix domain socket at "/tmp/localhost:60060". Yes, ":60060" is part of the socket's path name. Cute. Trying to use a more descriptive name like "goodsock" or whatever fails with "bad address". Hitting the GOODS source, unisock.cxx shows that that unix_socket_dir is hardcoded to "/tmp/", and that the file name format has to be "string:number".

Oh well. I've abstracted the procedure to obtain a Unix domain socket address from a path string from my previous post as follows:

NetNameResolver class>>addressForSocketPath: socketPath
  | size sa |

  NetNameResolver primGetAddressInfoHost: '' service: socketPath flags: 0 family: 1 type: 0 protocol: 0.
  size := NetNameResolver primGetAddressInfoSize.
  sa := SocketAddress new: size withAll: 0.
  NetNameResolver primGetAddressInfoResult: sa.
  ^ sa

However, asking for a Unix domain socket address for "/tmp/localhost:60060" causes primGetAddressInfoHost:blah:blah: to fail. Bummer.

Okay, it is easier to modify GOODS since I've been browsing its source, then to get well-acquainted with SocketPlugin. So, at line 107 of unisock.cxx, make this change:

//sprintf(u.name + offsetof(sockaddr,sa_data), "%s%s", unix_socket_dir, address);
sprintf(u.name + offsetof(sockaddr,sa_data), "%s%s", unix_socket_dir, hostname);

The commented out line is the original. "hostname" is "address" minus the colon and port number that comes after it.

Rebuild GOODS, change the path in test.cfg to say "goodserver:60060", restart, and we see that the Unix domain socket is now called "/tmp/goodserver" and "NetNameResolver addressForSocketPath: '/tmp/goodserver'" duly returns a SocketAddress instance.

Next, load the Squeak/Pharo GOODS client from SS3. Subclass KKSqueakTCPSocketTransport as KKPharoIPCSocketTransport, with the single method:

KKPharoIPCSocketTransport>>initializeSocketAddress: aSocketPath
  "Create and connect to specified Unix domain socket address."

  socket := Socket newIPC connectTo: (NetNameResolver addressForSocketPath: aSocketPath).
  (Delay forMilliseconds: 10) wait.

Some other corresponding modifications are needed, such as in KKDatabase and KKConnection. Finally, in workspace, run this:

| db |
KKDatabase defaultTransportClass: KKPharoIPCSocketTransport.
db := KKDatabase onSocketPath: '/tmp/goodserver'.
db root: Dictionary new.
db commit.
db explore.

And it works!

% goodsrv test 0 -
Read the whole database size to page pool
17:32.38 28-JUL-2013: Checkpoint 45 finished
GOODS server started...
server is up...
>
17:32.44 28-JUL-2013: Open session for client 'squeak16r38E5541'
Send class 'Dictionary' to client

Running "self logout" in the KKDatabase instance explorer from above results in additional output from GOODS:

17:39.23 28-JUL-2013: Client 'squeak16r38E5541' send logout request
17:39.23 28-JUL-2013: Disconnect 'squeak16r38E5541'
Server agent 'squeak16r38E5541' terminated
Tags: GOODS, OODB, unix domain sockets

Unix Domain Sockets on Pharo

27 Jul 2013

A Unix domain socket "or IPC (inter-process communication) socket is a data communications endpoint for exchanging data between processes executing within the same host operating system."

The Squeak/Pharo VM's socket plugin has had support for Unix domain sockets, as per SocketPrims.pdf, written by Ian Piumarta, dated 2007, "The old API assumes 32-bit INET4 (host) addresses and numbered service (port) addresses throughought. The new API supports arbitray host address sizes and both numbered and symbolic host and service names. [...] SQ_SOCKET_FAMILY_LOCAL The lookup should only find local (some- times called "Unix") domain host addresses. In this case, the host name should be empty and the service name should be a path to a node in the filesystem associated with a named socket."

And in code, platforms/unix/plugins/SocketPlugin/sqUnixSocket.c:

sqInt sqResolverGetAddressInfoFamily(void)
{
  if (!addrInfo)
    {
      interpreterProxy->success(false);
      return 0;
    }

  switch (addrInfo->ai_family)
    {
    case AF_UNIX:	return SQ_SOCKET_FAMILY_LOCAL;
    case AF_INET:	return SQ_SOCKET_FAMILY_INET4;
    case AF_INET6:	return SQ_SOCKET_FAMILY_INET6;
    }

  return SQ_SOCKET_FAMILY_UNSPECIFIED;
}

On the Smalltalk side, the Socket class has no support though. The key part is in the following fragment in "Socket>>initialize":

socketHandle := self
  primSocketCreateNetwork: 0
  type: socketType
  receiveBufferSize: 8000
  sendBufSize: 8000
  semaIndex: semaIndex
  readSemaIndex: readSemaIndex
  writeSemaIndex: writeSemaIndex.

Socket>>primSocketCreateNetwork:type:receiveBufferSize:sendBufSize:semdIndex: comments, "Return a new socket handle for a socket of the given type and buffer sizes. Return nil if socket creation fails. The netType parameter is platform dependent and can be used to encode both the protocol type (IP, Xerox XNS, etc.) and/or the physical network interface to use if this host is connected to multiple networks. A zero netType means to use IP protocols and the primary (or only) network interface."

Meaning, this bit "primSocketCreateNetwork: 0" hardcodes the use of TCP/IP. Easy enough to fix; let's rename Socket>>initialize to Socket>>initialize:withDomain:, changing the key part:

initialize: socketType withDomain: socketDomain

    [...]
    socketHandle := self
      primSocketCreateNetwork: socketDomain
      type: socketType
      receiveBufferSize: 8000
      sendBufSize: 8000
      semaIndex: semaIndex
      readSemaIndex: readSemaIndex
      writeSemaIndex: writeSemaIndex.
    [...]

Change Socket>>initialize to preserve the default behaviour:

initialize: socketType
  self initialize: socketType withDomain: 0

Next, create Socket class>>newIPC like this:

newIPC
  "Create a socket and initialise it for IPC aka Unix domain."
  self initializeNetwork.
  ^[ super new initialize: TCPSocketType withDomain: 1 ]
   repeatWithGCIf: [ :socket | socket isValid not ]

Here "TCPSocketType" really means SOCK_STREAM in the Unix C API parlance.

We'll need Socket>>connectTo: as well:

connectTo: socketAddress
  | status |
   
  status := self primSocketConnectionStatus: socketHandle.
  (status == Unconnected)
    ifFalse: [InvalidSocketStatusException signal: 'Socket status must Unconnected before opening a new connection'].
   
  self primSocket: socketHandle connectTo: socketAddress

Now, how to get a Unix domain socket address? After some experimentation, this works:

NetNameResolver primGetAddressInfoHost: '' service: '/tmp/sp' flags: 0 family: 1 type: 0 protocol: 0.
s := NetNameResolver primGetAddressInfoSize.
sa := SocketAddress new: s withAll: 0.
NetNameResolver primGetAddressInfoResult: sa.

To test, use some known-working Unix domain socket test programs. The following is a slightly modified, rather self-explanatory Python server from here:

import socket,os
sp = "/tmp/sp"
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
    os.remove(sp)
except OSError:
    pass
s.bind(sp)
s.listen(1)
conn, addr = s.accept()
while 1:
    data = conn.recv(1024)
    if not data: break
    print data
    conn.send(data)
conn.close()

There is a corresponding client program. Run the two to verify that the server works.

Back to Smalltalk, the following code run in a workspace will connect to above Python server, send, receive, then print "Hello, from Smalltalk!" on the transcript. Tested on OSX and Linux with Pharo 2.0.

| s sa sock |
NetNameResolver primGetAddressInfoHost: '' service: '/tmp/sp' flags: 0 family: 1 type: 0 protocol: 0.
s := NetNameResolver primGetAddressInfoSize.
sa := SocketAddress new: s withAll: 0.
NetNameResolver primGetAddressInfoResult: sa.
sock := Socket newIPC.
sock ifNotNil: [
    sock connectTo: sa.
    sock sendData: 'Hello, from Smalltalk!'.
    Transcript show: sock receiveData; cr.
    sock closeAndDestroy ]
Tags: unix domain sockets