\documentclass[a4paper,10pt]{article} \usepackage[compat2]{geometry} \usepackage{url} \usepackage{hyperref} \makeatletter \def\s@btitle{\relax} \def\subtitle#1{\gdef\s@btitle{#1}} \def\@ddress{\relax} \def\address#1{\gdef\@ddress{#1}} \def\@maketitle{% \newpage \null \vskip 2em% \begin{center}% \let \footnote \thanks {\LARGE \@title \par}% \if\s@btitle\relax \else\typeout{[subtitle]}% \vskip .5pc \begin{large}% \textsl{\s@btitle}% \par \end{large}% \fi \vskip 1.5em% {\large \lineskip .5em% \begin{tabular}[t]{c}% \@author \end{tabular}\par}% \vskip 1em% \if\@ddress\relax \else\typeout{[address]}% \vskip .5pc \begin{large} \textsl{\@ddress}% \par \end{large} \fi \vskip 2em% {\large \@date}% \end{center}% \par \vskip 1.5em} \makeatother \title{Libre Software Meeting 2004} \subtitle{Proceedings of the VHLL track} \author{Christophe Rhodes (ed.)\\\texttt{}} \date{8 July 2004} \begin{document} \maketitle \setcounter{tocdepth}{1} \tableofcontents \section*{Introduction} The following are summaries of the lightning talks held on 8 July 2004 at the Libre Software Meeting 2004, held at ENSEIRB, Bordeaux. The talks themselves were informal, as are these proceedings: the intent was more to air half-formed ideas as a basis for discussion, rather than to present finished work; nevertheless, it is hoped that these ideas may be of interest to the wider Common Lisp community. \begin{flushright} Christophe Rhodes, 2 August 2004 (version 1) \end{flushright} \clearpage \section{SBCL Threads} \begin{center} Daniel Barlow\\ \verb++ \end{center} SBCL includes support for OS-scheduled threads, which are therefore SMP-capable but do not allow Lisp control of the scheduler. This requires x86 and Linux kernel 2.6 or systems with NPTL backports. \subsection{Special variables} The interaction of special variables with multiple threads is mostly as expected, but in places somewhat different from that traditionally implemented e.g. in Allegro CL, Lispworks etc.: \begin{itemize} \item global special values are visible across all threads; \item bindings (e.g. using LET) are local to the thread; \item initial values in a new thread are taken from the thread that created it. \end{itemize} \subsection{Mutex support} For controlling access to a shared resource. One thread is allowed to hold the mutex, others which attempt to take it will be made to wait until it's free. Threads are woken in the order that they go to sleep There isn't a timeout on mutex acquisition, but the usual WITH-TIMEOUT macro (which throws a TIMEOUT condition after n seconds) can be used if you want a bounded wait \begin{verbatim} (defpackage :demo (:use "CL" "SB-THREAD" "SB-EXT")) (in-package :demo) (defvar *a-mutex* (make-mutex :name "my lock")) (defun thread-fn () (let ((id (current-thread-id))) (format t "Thread ~A running ~%" id) (with-mutex (*a-mutex*) (format t "Thread ~A got the lock~%" id) (sleep (random 5))) (format t "Thread ~A dropped lock, dying now~%" id))) (make-thread #'thread-fn) (make-thread #'thread-fn) \end{verbatim} \subsection{Waitqueue/condition variables} These are based on the POSIX condition variable design, hence the annoyingly CL-conflicting name. For use when you want to check a condition and sleep until it's true. For example: you have a shared queue, a writer process checking ``queue is empty'' and one or more readers that need to know when ``queue is not empty''. It sounds simple, but is astonishingly easy to deadlock if another process runs when you weren't expecting it to. There are three components: \begin{itemize} \item the condition itself (not represented in code) \item the condition variable (a.k.a waitqueue) which proxies for it \item a lock to hold while testing the condition \end{itemize} Important stuff to be aware of: \begin{enumerate} \item when calling \texttt{condition-wait}, you must hold the mutex. \texttt{condition-wait} will drop the mutex while it waits, and obtain it again before returning for whatever reason; \item likewise, you must be holding the mutex around calls to \texttt{condition-notify}; \item a process may return from \texttt{condition-wait} in several circumstances: it is not guaranteed that the underlying condition has become true. You must check that the resource is ready for whatever you want to do to it. \end{enumerate} \begin{verbatim} (defvar *buffer-queue* (make-waitqueue)) (defvar *buffer-lock* (make-mutex :name "buffer lock")) (defvar *buffer* (list nil)) (defun reader () (with-mutex (*buffer-lock*) (loop (condition-wait *buffer-queue* *buffer-lock*) (loop (unless *buffer* (return)) (let ((head (car *buffer*))) (setf *buffer* (cdr *buffer*)) (format t "reader ~A woke, read ~A~%" (current-thread-id) head)))))) (defun writer () (loop (sleep (random 5)) (with-mutex (*buffer-lock*) (let ((el (intern (string (code-char (+ (char-code #\A) (random 26))))))) (setf *buffer* (cons el *buffer*))) (condition-notify *buffer-queue*)))) (make-thread #'writer) (make-thread #'reader) (make-thread #'reader) \end{verbatim} \subsection{Miscellany} \begin{itemize} \item the debugging interface is fairly rudimentary. Threads that want to debug will print a message on the terminal to say so, then wait on a lock for the user's input. The user may select the next thread to debug by using \texttt{(release-foreground)} to make the current thread release this lock -- then the debugger will appear as normal. \item interrupt-thread works, but is not advisable anyway: the code you're interrupting may not expect to be stopped. \item some parts of the system aren't thread-safe: we don't know which... \begin{itemize} \item the compiler and loader are protected by a big lock, so safe (though serialised); \item in other areas (maybe streams or the printer) you may lose. \end{itemize} \item the plan is to layer a high-level interface (something similar to CLIM-SYS) on top of this\footnote{See \url{http://cvs.telent.net/cgi-bin/viewcvs.cgi/bordeaux-mp/Specification?view=markup}.}. \end{itemize} \clearpage \section{ASDF/ASDF-INSTALL} \begin{center} Daniel Barlow\\ \verb++ \end{center} \subsection{Using asdf-install} \begin{verbatim} (require 'asdf-install) (asdf-install:install 'split-sequence) \end{verbatim} downloads, compiles and installs the \texttt{SPLIT-SEQUENCE} package, as well as any packages it depends on that are not already installed. It works using specially formatted links on CLiki pages: to asdf-install the foo package, \url{www.cliki.net/foo} is expected to contain a link \begin{verbatim} :(package "http://www.example.com/foo.tar.gz") \end{verbatim} to the location from which {\tt foo.tar.gz} and {\tt foo.tar.gz.asc} (see below) can be downloaded. \subsection{GNU Privacy Guard} Anybody can change a CLiki page, which may cause you to load arbitrary code into your lisp. This is usually considered Not Good, so we do crypto signature checking using the GNU Privacy Guard. asdf-install performs three checks: \begin{enumerate} \item The signature exists; \item You trust that the signer is who he says he is: a GPG trust relationship exists between you and him; \item You trust the signer's good intentions and Lisp abilities: asdf-install maintains a list of Lisp hackers' fingerprints in \texttt{\$HOME/.sbcl/trusted-uids.lisp}. \end{enumerate} Every time you meet a Lisp hacker, exchange GPG keys with him. Likewise Debian developers, who have invested serious time into their PGP web of trust so are great people to piggyback off. \subsection{Writing your own packages} \begin{enumerate} \item Write an asdf system definition that knows how to compile/load the system. There is documentation (hidden in a \texttt{README} file), but most people copy an existing one. \item Tar it up. \item Create a detached GPG signature. \item Upload both files somewhere. \item And add a link on CLiki. Finito. \end{enumerate} More information is available from \url{http://www.cliki.net/asdf-install} \clearpage \section{Making McCLIM use less ugly fonts} \begin{center} Andreas Fuchs\\ \verb++ \end{center} Using a typical CLIM application in the \url{http://clim.mikemac.com/} McCLIM means putting up with a Courier font for all text (unless the application specified the use of another text family for output). Changing that involves either hacking (and recompiling) the application or hacking (and, expensively, recompiling) McCLIM - both of which aren't optimal nor interesting solutions, especially for new users of McCLIM. I present a way of making McCLIM use a different default font for text that does not involve recompilation of anything. Unfortunately, this method isn't very good lisp code either, but it's a drop-in solution that works for now. (The McCLIM developers that were present during the talk agreed that it would be a better idea to make the fonts settable at run-time via a sane mechanism.) What happens is this: The CLX backend defines mappings of CLIM font names to actual X fonts. These are set up for the CLX port when the first display is opened. So, the code at\\ \url{http://boinkor.net/lisp/font-hackery.lisp} defines an :around method for the font initialization generic function, and rebinds the default font mapping to something that is more desirable to the user. \clearpage \section{Case sensitive Common Lisp packages} \begin{center} Bruno Haible\\ \verb++ \end{center} This talk presents a way to allow Common Lisp to be programmed in a case-sensitive way, while at the same time allowing seamless integration with legacy Common Lisp programs that assume case-insensitive behaviour. The goal is to make possible Common Lisp programs with case sensitive symbols. Example: \begin{verbatim} > (list 'MacOS 'X) (MacOS X) \end{verbatim} There are implementations of this feature (Franz Inc.'s Allegro CL), but they don't allow legacy Common Lisp programs to be used at the same time, thus creating a schism in the CL world and posing a major obstacle to the adoption of this modern style. Here an approach is presented that allows case-sensitive packages and case-insensitive packages to coexist in the same process and interoperate with each other. Example: \begin{verbatim} OLD.LISP (IN-PACKAGE "OLD") (DEFUN FOO () ...) modern.lisp (in-package "NEW") (defun bar () (old:foo)) (symbol-name 'bar) => "bar" \end{verbatim} \subsection{How it works} \begin{enumerate} \item There is the notion of case-sensitive packages. When a package is declared or created, it can take the option \texttt{:case-sensitive t.} The effect of this declaration is that the reader doesn't uppercase the symbol name before calling intern. Similarly, the printer, when printing the symbol name part of a symbol (i.e. the part after the package markers), behaves as if the readtable's case were set to \texttt{:preserve}. \item There is the notion of case-inverted packages. When a package is declared or created, it can take the option \texttt{:case-inverted t}. What this means, is that in the context of such a package, symbol names are case-inverted: upper case characters are mapped to lower case, lower case characters are mapped to upper case, and other characters are left untouched. (Mapping lower case characters to upper case is not particularly desirable, but it doesn't matter since symbol names with lower case characters are rare in the ``old'' world. However, what matters is: It is essential that the mapping be injective and covariant w.r.t. string concatenation. The chosen case-flipping satisfies this.) Every symbol thus conceptually has two symbol names: an old-world symbol name and a new-world symbol name, which is the case-inverted old-world name. The first symbol name is returned by the function \texttt{CL:SYMBOL-NAME}, the modern one by the function \texttt{ci-cl:symbol-name}. An implementation may choose to store only one of these two symbol names, and compute the other one on the fly. The two package invariants involving strings hold whether you view them using \texttt{CL:SYMBOL-NAME} or \texttt{ci-cl:symbol-name}: \begin{itemize} \item Let $s$ be a given string and $p$ a package. If the set of accessible symbols of $p$ which have the print name $s$ has cardinality $> 1$, then exactly one of these symbols is a shadowing symbol of $p$. \item Let $s$ be a given string and $p$ a package. The set of interned symbols of $p$ (exported or not) which have the print name $s$ has cardinality $\leq 1$. \end{itemize} Therefore the package system continues to work either way. The internal functions for creating or looking up symbols in a package, which traditionally took a string argument, now conceptually take two string arguments: old-style-string and inverted-string. (Again an implementation is free to choose a different internal working, such as either always passing old-style-string, or always passing inverted-string, or passing string and invertedp arguments.) For a few built-in functions, a variant for the case-inverted world needs to be defined: \begin{itemize} \item \texttt{ci-cl:symbol-name} returned the case-inverted symbol name, \item \texttt{ci-cl:intern} and \texttt{ci-cl:find-symbol} need to work consistently with \texttt{ci-cl:symbol-name}, \item \texttt{ci-cl:shadow}, \texttt{ci-cl:find-all-symbols}, and string comparison and trimming operators may need at some point to convert a symbol to a string and therefore exist in a variant that uses \texttt{ci-cl:symbol-name} instead of \texttt{CL:SYMBOL-NAME}. \item \texttt{ci-cl:make-package} creates a case-inverted package. \end{itemize} These variants, together with the unmodified symbols from the \texttt{COMMON-LISP} package, can conveniently be exported from a \texttt{CASE-SENSITIVE-COMMON-LISP} package, with nicknames \texttt{CS-CL} and \texttt{CI-CL}. Similarly, a package \texttt{CS-CL-USER} playing the same role as \texttt{CL-USER}, but for the case-sensitive world, can be provided. \item The handling of package names is unchanged. Package names are still usually uppercase. \end{enumerate} Note that gensyms and keywords are still treated traditionally: even in a case-sensitive package, \texttt{(eq \#:FooBar \#:foobar)} and \texttt{(eq ':KeyWord ':keyword)}. We believe this has limited negative impact for the moment, and can be changed in a second step, a few years from now. \subsection{Migration tips} The following practices will pose no problems when migrating to a modern case-sensitive world: \begin{itemize} \item Using CL symbols in lowercase, \item Macros that create symbols by suffixing or prefixing given symbols, \item Comparing symbol names as in \texttt{(string= (symbol-name x) (symbol-name y))}. \end{itemize} The following practices will not work in a case-sensitive world or can give problems: \begin{itemize} \item Accessing the same symbol in both upper- and lowercase from the same source file. \item Macros that create symbols in other packages than the original symbols. \item Comparing symbol-name return values with \texttt{eq}. \item Comparing \texttt{(cl:symbol-name x)} with \texttt{(ci-cl:symbol-name y)}. \end{itemize} \clearpage \section{Using the MOP for a Foreign Language Interface to Java} \begin{center} Bruno Haible\\ \verb++ \end{center} This talk explains how the MOP can be used to define the critical parts of an object-oriented foreign language interface in a portable way. We assume at the basis that we have a Common Lisp implementation that can \begin{enumerate} \item access Java objects, \item call Java functions and methods, \item generate Java bytecoded classes. \end{enumerate} Note: Java and CL can be tied in different ways: \begin{itemize} \item CL implemented in Java (like the Armed Bear CL from Peter Graves), \item CL and Java operating in the same process, or \item CL and Java in separate processes, using socket communication. \end{itemize} The precise nature of the Java / CL coupling is not relevant here. The goal of the project would be create subclasses of Java classes like this: \begin{verbatim} (defclass (javax.swing.JComponent) ((hello-label :type javax.swing.JLabel)) (:metaclass java-class)) (defmethod paintComponent ((c ) (g java.awt.Graphics)) ...) \end{verbatim} The MOP is used in three areas. \subsection{Class Definition Customization} \texttt{compute-superclasses} is modified 1. to take into account \texttt{java.lang.Object} as superclass when no superclass is specified. 2. to ensure a proxy class for redirecting Java public/protected methods into Lisp. (Namely, every time a Lisp class $L$ is defined as being a subclass of a Java class $J$, under the hood, a Java subclass $J'$ of $J$ is defined, and $L$ is defined as inheriting from $J'$, not $J$.) \texttt{compute-slots} has a modified \texttt{:around} method to allocate slots in Java objects vs. its Lisp counterpart, taking into account the fact that in Java, slots of type `double' take two words instead of just one (assuming 32-bit words). \subsection{Slot Access Customization} \texttt{slot-value-using-class} is modified to take into account whether a slot is allocated on the Java side or on the Lisp side of an object. \subsection{Method Customization} A class \texttt{java-method}, subclass of \texttt{standard-method}, is created with a \texttt{:function} argument, that invokes a Java method. This is necessary to make Java methods visible as first-class objects in the Lisp world. \clearpage \section{Efficiently handling multiple network clients} \begin{center} \'{E}ric Marsden\\ \verb++ \end{center} \subsection{Introduction} \subsubsection{Context: the c10k problem} How can we service 10000 concurrent clients from a single lisp image? \begin{itemize} \item web servers \item IMAP servers \end{itemize} \textsl{(Even 1000 clients is a challenge!)} Problems: \begin{itemize} \item each connection has a state that must be maintained \item a greedy or slow client should not starve other clients \item should handle communication errors in a robust way \item each connection should be serviced incrementally \end{itemize} \subsubsection{Potential I/O strategies} Two main concurrency strategies: \begin{itemize} \item \textbf{threads}: programmer writes straight-line code and the OS interleaves computations by switching between threads \begin{itemize} \item large numbers of threads generally scale badly \end{itemize} \item \textbf{events}: programmer declares handlers that are called upon I/O completion or I/O readiness or timer expiration \begin{itemize} \item generally less straightforward to program \item partially complete requests must be buffered \end{itemize} \end{itemize} Combined with various I/O strategies: \begin{itemize} \item non-blocking I/O \item readiness change notification (\texttt{select} system call) \item asynchronous I/O \end{itemize} \subsection{SERVE-EVENT} \subsubsection{CMUCL's SERVE-EVENT mechanism} CMUCL includes a \textsc{serve-event} mechanism that is quite well designed and convenient to use. \begin{itemize} \item select-based readiness notification mechanism \item makes it possible to service multiple clients \begin{itemize} \item without resorting to threads! \end{itemize} \item well integrated with CMUCL: \begin{itemize} \item the I/O subsystem \item the listener \item CLX \end{itemize} \item user registers a handler that is called when activity is detected on a given file descriptor \end{itemize} API: \begin{itemize} \item \texttt{(system:add-fd-handler :input (lambda (fd) ...))} \item \texttt{(system:remove-fd-handler )} \item \texttt{(system:invalidate-descriptor )} \item \texttt{(system:serve-event \&optional timeout)} \end{itemize} \subsection{Hazards} \subsubsection{SERVE-EVENT traps} \begin{itemize} \item how to handle buffering? \item how to handle read timeouts? \item limited to watching \texttt{FDSET\_SIZE} descriptors \begin{itemize} \item could be fixed by calling \texttt{poll} instead of \texttt{select} \end{itemize} \item an event handler that doesn't handle all errors can affect other handlers \item recursive reentry of \texttt{serve-event} via na\"ive use can cause starvation (example follows) \end{itemize} \subsubsection{Luke Gorrie's delay exploit} \begin{itemize} \item Example service that is open to abuse by malicious or hung clients \item Uses \texttt{read} to handle connections from multiple clients \item The reentry of \texttt{serve-event} upon blocking input in \texttt{read} will lead to interference between clients \end{itemize} Here is the handler that is installed by \texttt{add-fd-handler} and called upon activity on this socket: \begin{verbatim} (defun server-handle-connection (number socket) (let ((stream (sys:make-fd-stream socket :input t)) (successful nil)) (unwind-protect (with-standard-io-syntax (say "Connection #~D read ~A" number (read stream)) (setq successful t)) (close-connection stream) (unless successful (say "Connection #~D aborted!" number))))) \end{verbatim} Here is a case with two clients, $A$ and $B$, where $A$ must wait for $B$ to finish a request before being able to proceed, even though all of $A$'s data is available to the server. \begin{verbatim} (defun delay () (with-clients (a b) (send a first-half) (send b first-half) (send a second-half) (sleep 5) (send b second-half))) \end{verbatim} Both clients connect and send half of a request, with $A$ sending first. The server enters ``blocking'' READ, first for $A$ and then for $B$, and awaits more input. The relevant parts of the server's Lisp stack look like this: \begin{verbatim} (SERVE-EVENT) (READ B) (SERVE-EVENT) (READ A) (SERVE-EVENT) (SERVER-LOOP) \end{verbatim} Next $A$ sends the rest of his request. But what can we do with it? Nothing yet: we cannot return from \texttt{(READ A)} without first returning from \texttt{(READ B)}, and $B$ is still blocking. $A$ must wait for $B$'s request to complete. In the test case this takes a few seconds. This case can occur if the network fails between client and server, and it's easy to trigger deliberately (maliciously). \textbf{Partial solution}: timeouts on streams \subsubsection{Error propagation exploit} Another issue presents itself from looking at the previous stack diagram. What if an unhandled condition is signalled in the \texttt{read} of $B$? With our server it will propagate right up the stack and be handled by \texttt{server-loop}. That means that an error triggered by client $B$ will cause client $A$'s handler to be unwound from the stack, aborting his connection in the process. \textbf{Fix}: always handle all errors in an event handler \subsection{Framework} \subsubsection{Safe use of serve-event} Use a framework that decouples I/O handling from request processing: \begin{itemize} \item avoid reentry of serve-event by never reading more data than is available \item on each connection, read available data into a buffer in a non-blocking manner until you have a complete request \item add the request to a queue \item in the event loop, handle requests from the queue (without blocking!) \item write response to client, without blocking \end{itemize} Limitations: \begin{itemize} \item only efficient for request/reply protocols such as HTTP (inefficient for streaming protocols such as chargen) \item only works for TCP connections (no UDP) \end{itemize} Code for the framework is available at \url{http://www.laas.fr/~emarsden/etc/event-server.lisp}. Stress-tested using seige: limited by 100 Mb/sec Ethernet \subsubsection{Future work} For non-blocking I/O, we use a CMUCL-specific function named \texttt{sys:read-n-bytes} instead of the standard \texttt{read-sequence}. The CL standard does not specify any way of doing non-blocking I/O. This function reads directly from a file descriptor, so it doesn't cooperate well with libraries like the \textit{cmucl-ssl} bindings to OpenSSL that also require access to file descriptors (in the case of \textit{cmucl-ssl}, replacing fd-stream functions by FFI calls to the OpenSSL library functions \texttt{SSL\_read} and \texttt{SSL\_write}). Simple-streams might be a nice way to do non-blocking I/O in a semi-standardized fashion. For very high performance, it would be good to avoid copying of data in lisp and lisp $\rightarrow$ TCP (zero-copy issues, requiring interaction with the operating system). \clearpage \section{:IEEE-FLOATING-POINT} \begin{center} Christophe Rhodes\\ \verb++ \end{center} The \texttt{:ieee-floating-point} feature in a Common Lisp implementation \begin{quote} if present, indicates that the implementation purports to conform to the requirements of {\it IEEE Standard for Binary Floating Point Arithmetic}. -- the ANSI Common Lisp standard \end{quote} IEEE 754 only specifies the results of basic operations: in Common Lisp terms, the arithmetic \texttt{+}, \texttt{-}, \texttt{*}, \texttt{/}, and \texttt{sqrt} operators, the rounding operators, and coercion. However, it specifies not only the return values, but also the effect on the floating point state: certain operations are specified to raise exceptions (which can be selectively masked). For instance, \verb+(/ 1.0 0.0)+ should return single float positive infinity, but should also raise the \texttt{division-by-zero} exception. This is relatively uncontroversial in the Common Lisp world -- though implementations differ over whether the trap results in an Lisp-level condition being signalled. However, the IEEE 754 standard also suggests that, if possible, trapping operations should provide sufficient context for the user to be able to know the causes of the trap: passing operands and the attempted operation to any condition handler. At present, no Common Lisp implementation does this to a meaningful extent. There are other, conceptually simpler, disconnects between the requirements of IEEE 754 and Common Lisp, such as the result of \verb+(sqrt -1.0)+; Common Lisp integrates complex numbers fully with its arithmetic operations, and sqrt is defined to return the prinicipal value, whereas IEEE 754 prescribes a NaN return value and the \verb+:invalid+ trap. If there is one thing clear about the :ieee-floating-point keyword feature, then, it is that its meaning is unclear; a simple meaning that current practice is compatible with is that the basic operations on floating point numbers give IEEE 754-compatible return values. However, a best-faith attempt to implement more advanced semantics would probably have value. In a situation where the intended meanings are unclear, it is often preferable to attempt to write tests first, as this firstly establishes what interfaces are necessary, and secondly clarifies assumptions. A project for testing IEEE floating point semantics has been started\footnote{see \url{http://www.common-lisp.net/project/ieeefp-tests} for more information.} and includes translations of previous testing attempts (primarily for the Fortran and C languages) into Common Lisp, with some implementation-specific support code which it is hoped will form the nucleus of an IEEE floating point interface. Some ``advanced'' features of IEEE floating point semantics are demonstrably possible to support: various Common Lisp implementations have interfaces allowing the user to control the rounding mode, the precision or masking of trap extensions (such as the x86's denormalized-operand trap). There remain, however, unanswered questions, such as whether the behaviour of \texttt{round} (or \texttt{fround}, or \texttt{ftruncate}) is sensitive to the FPU rounding mode; whether the reader is sensitive to the rounding mode (so that, for instance, \verb+(read-from-string "1.0e9999999999")+ should return single-float-positive-infinity if the rounding mode is towards negative infinity), and whether it makes sense to specify traps taken during the computation of transcendental functions. \clearpage \section{Some things about XML-RPC} \begin{center} Rudi Schlatte\\ \verb++ \end{center} \subsection{What is it?} xml-rpc is a lightweight remote procedure call specification that uses HTTP as transport protocol; the payload is encoded with XML. The (short) spec resides at \url{http://www.xmlrpc.com/spec}. xml-rpc has a fixed set of scalar datatypes: \begin{itemize} \item integer (signed 32-bit) \item boolean \item string \item double \item Date/Time (iso8601 w/o timezone) \item base64-encoded binary value \end{itemize} Composite datatypes are array and struct (associative array of string to any value). Arrays and structs are dynamically-typed-language-friendly since they don't force all their elements to be of the same type. \subsection{Why should we care?} xml-rpc is easy to understand and use, and libraries exist for multiple languages. As mentioned, it is also friendly to dynamically-typed languages. It is used in the blogging community for offline weblog editors (\url{http://www.xmlrpc.com/directory/1568/bloggingApis}). The lisppaste bot at \url{http://paste.lisp.org/} can be driven over xml-rpc as well. \url{http://common-lisp.net/project/lisppaste/lisppaste.el} is a snippet of code that defines an Emacs `lisppaste-region' command. \subsection{How is it used?} The following examples use the s-xml-rpc library at \url{http://www.common-lisp.net/project/s-xml-rpc/}. \subsubsection{Client side:} \begin{verbatim} (call-xml-rpc-server '(:host "betty.userland.com") "examples.getStateName" 41) => "South Dakota" \end{verbatim} \subsubsection{Server side:} \begin{verbatim} (defun add2 (x y) (+ x y)) (import 'add2 s-xml-rpc:*xml-rpc-package*) \end{verbatim} \subsection{Any other things?} There are some layered semi-standards, the most useful of which are: \begin{itemize} \item server introspection \url{http://xmlrpc.usefulinc.com/doc/reserved.html} \item multicall (multiple calls per HTTP roundtrip) \url{http://www.xmlrpc.com/discuss/msgReader$1208}%$ \end{itemize} \clearpage \section{Problems with extensible streams} \begin{center} Rudi Schlatte\\ \verb++ \end{center} \subsection{Motivation for extensible streams} We want something like this: \begin{verbatim} (make-instance 'db-stream :select-string "select blob from movies where title = 'xXx'") \end{verbatim} or \begin{verbatim} (make-instance 'socket-stream :remote-host "google.com" :remote-port 80) \end{verbatim} ... and be able to use \texttt{read-sequence} on the resulting object. The two existing approaches are Gray streams\footnote{\url{http://www.nhplace.com/kent/CL/Issues/stream-definition-by-user.html}} and simple-streams\footnote{\url{http://www.franz.com/support/documentation/6.2/doc/streams.htm}}, each with its own set of problems. \subsection{Gray streams} Gray streams have the virtue of being supported by almost all CL implementations currently in use. The disadvantage of Gray streams is that subclassing can lose in interesting ways. An example would be a mixin class that keeps track of the column position of a stream: \begin{verbatim} (defmethod stream-read-char :around ((stream column-counting-mixin)) (let ((result (call-next-method))) (if (char= result #\Newline) (setf column-position 0) (incf column-position)) result)) \end{verbatim} The equivalent function either must or must not be implemented for \texttt{stream-read-char-no-hang}, depending on whether the stream class that is extended via this mixin implements read-char-no-hang by calling read-char or not. There is no way to write this mixin in a general way! Another issue is that Gray streams don't define \texttt{stream-read-sequence} and \texttt{stream-write-sequence}; CL implementations differ in the way they make this functionality available to the user. Finally, the fact that every stream operation involves a generic function call might or might not be a performance problem. \subsection{simple-streams} simple-streams make the following assumptions: \begin{itemize} \item The outside world (that we want to interface with) consists of streams and blobs of octets \item All streams are buffered \item Stream extension happens at the device level, not at the CL stream API (this avoids the stream-read-char problem) \end{itemize} simple-streams originated in Allegro Common Lisp; a partial implementation is available in cmucl, and in sbcl (in the sb-simple-streams contrib module). Sadly, simple-streams are underspecified in places (notably, external-format handling), and the device protocol for implementing new types of streams can be described as ``hairy''. Also, some details of the specification seem to be mandated by implementation needs. \subsection{Perspective} As a result of discussions at the Libre Software Meeting, a mailing list was created for further discussions of extensible streams\footnote{see \url{http://common-lisp.net/mailman/listinfo/streams-standard-discuss}.}. \end{document}