Changes in PyZMQ
This is a coarse summary of changes in pyzmq versions. For a real changelog, consult the
Minor bugfixes to pyzmq 14.3:
- Fixes to building bundled libzmq on OS X < 10.9
- Fixes to import-failure warnings on Python 3.4
- Fixes to tests
- Pull upstream fixes to zmq.ssh for ssh multiplexing
- PyZMQ no longer calls Socket.close() or Context.term() during process cleanup.
Changes to garbage collection in Python 3.4 make this impossible to do sensibly.
- ZMQStream.close() closes its socket immediately, rather than scheduling a timeout.
- Raise the original ImportError when importing zmq fails.
Should be more informative than no module cffi....
Users of Python 3.4 should not use pyzmq < 14.3, due to changes in garbage collection.
- Raise new ZMQVersionError when a requested method is not supported by the linked libzmq.
For backward compatibility, this subclasses NotImplementedError.
- Memory leak introduced in pyzmq-14.0 in zero copy.
- OverflowError on 32 bit systems in zero copy.
The headline features for 14.1 are adding better support for libzmq’s
- When libzmq is bundled as a Python extension (e.g. wheels, eggs),
libsodium is also bundled (excluding Windows),
ensuring that libzmq security is available to users who install from wheels
- New zmq.auth, implementing zeromq’s ZAP authentication,
modeled on czmq zauth.
For more information, see the examples.
Other New Stuff
- Add PYZMQ_BACKEND for enabling use of backends outside the pyzmq codebase.
- Add underlying property and shadow()
method to Context and Socket, for handing off sockets and contexts.
between pyzmq and other bindings (mainly pyczmq).
- Add TOS, ROUTER_HANDOVER, and IPC_FILTER constants from libzmq-4.1-dev.
- Add Context option support in the CFFI backend.
- Various small unicode and build fixes, as always.
- send_json() and recv_json() pass any extra kwargs to json.dumps/loads.
- Socket.socket_type is deprecated, in favor of Socket.type,
which has been available since 2.1.
- Update bundled libzmq to current (4.0.3).
- Fix bug in Context.destroy() with no open sockets.
- Threadsafety fixes in the garbage collector.
- Python 3 fixes in zmq.ssh.
- Update bundled libzmq to current (4.0.1).
- Backends are now implemented in zmq.backend instead of zmq.core.
This has no effect on public APIs.
- Various build improvements for Cython and CFFI backends (PyPy compiles at build time).
- Various GIL-related performance improvements - the GIL is no longer touched from a zmq IO thread.
- Adding a constant should now be a bit easier - only zmq/sugar/constant_names should need updating,
all other constant-related files should be automatically updated by setup.py constants.
- add support for latest libzmq-4.0.1
(includes ZMQ_CURVE security and socket event monitoring).
The main new feature is improved tornado 3 compatibility.
PyZMQ ships a ‘minitornado’ submodule, which contains a small subset of tornado 3.0.1,
in order to get the IOLoop base class. zmq.eventloop.ioloop.IOLoop is now a simple subclass,
and if the system tornado is ≥ 3.0, then the zmq IOLoop is a proper registered subclass
of the tornado one itself, and minitornado is entirely unused.
A few things were broken in 13.0.0, so this is a quick bugfix release.
- FIXED EAGAIN was unconditionally turned into KeyboardInterrupt
- FIXED we used totally deprecated ctypes_configure to generate constants in CFFI backend
- FIXED memory leak in CFFI backend for PyPy
- FIXED typo prevented IPC_PATH_MAX_LEN from ever being defined
- FIXED various build fixes - linking with librt, Cython compatibility, etc.
defunct bugfix. We do not speak of this...
PyZMQ now officially targets libzmq-3 (3.2.2),
0MQ ≥ 2.1.4 is still supported for the indefinite future, but 3.x is recommended.
PyZMQ has detached from libzmq versioning,
and will just follow its own regular versioning scheme from now on.
PyZMQ bdists will include whatever is the latest stable libzmq release (3.2.2 for pyzmq-13.0).
set/get methods are exposed via get/setattr on all Context, Socket, and Frame classes.
This means that subclasses of these classes that require extra attributes
must declare these attributes at the class level.
- The Threadsafe ZMQStream experiment in 18.104.22.168 was deemed inappropriate and not useful,
and has been removed.
- The zmq.web experiment has been removed,
to be developed as a standalone project.
- Support for PyPy via CFFI backend (requires py, ctypes-configure, and cffi).
- Add support for new APIs in libzmq-3
- Setting and getting Socket.hwm sets or gets both SNDHWM/RCVHWM for libzmq-3.
- Implementation splits core Cython bindings from pure-Python subclasses
with sugar methods (send/recv_multipart). This should facilitate
non-Cython backends and PyPy support [spoiler: it did!].
- Unicode fixes in log and monitored queue
- MinGW, ppc, cross-compilation, and HP-UX build fixes
- zmq.green should be complete - devices and tornado eventloop both work
in gevent contexts.
This is a tech-preview release, to try out some new features.
It is expected to be short-lived, as there are likely to be issues to iron out,
particularly with the new pip-install support.
Experimental New Stuff
These features are marked ‘experimental’, which means that their APIs are not set in stone,
and may be removed or changed in incompatible ways in later releases.
With the IOLoop inherited from tornado, there is exactly one method that is threadsafe:
IOLoop.add_callback(). With this release, we are trying an experimental option
to pass all IOLoop calls via this method, so that ZMQStreams can be used from one thread
while the IOLoop runs in another. To try out a threadsafe stream:
stream = ZMQStream(socket, threadsafe=True)
pip install pyzmq
PyZMQ should now be pip installable, even on systems without libzmq.
In these cases, when pyzmq fails to find an appropriate libzmq to link against,
it will try to build libzmq as a Python extension.
This work is derived from pyzmq_static.
To this end, PyZMQ source distributions include the sources for libzmq (2.2.0) and libuuid (2.21),
both used under the LGPL.
- TIMEO sockopts are properly included for libzmq-2.2.0
- avoid garbage collection of sockets after fork (would cause assert (mailbox.cpp:79)).
Some effort has gone into refining the pyzmq API in this release to make it a model for
other language bindings. This is principally made in a few renames of objects and methods,
all of which leave the old name for backwards compatibility.
As of this release, all code outside zmq.core is BSD licensed (where
possible), to allow more permissive use of less-critical code and utilities.
- The Message class has been renamed to Frame, to better match other
zmq bindings. The old Message name remains for backwards-compatibility. Wherever pyzmq
docs say “Message”, they should refer to a complete zmq atom of communication (one or
more Frames, connected by ZMQ_SNDMORE). Please report any remaining instances of
Message==MessagePart with an Issue (or better yet a Pull Request).
- All foo_unicode methods are now called foo_string (_unicode remains for
backwards compatibility). This is not only for cross-language consistency, but it makes
more sense in Python 3, where native strings are unicode, and the _unicode suffix
was wedded too much to Python 2.
Other Changes and Removals
- prefix removed as an unused keyword argument from send_multipart().
- ZMQStream send() default has been changed to copy=True, so it matches
- ZMQStream on_err() is deprecated, because it never did anything.
- Python 2.5 compatibility has been dropped, and some code has been cleaned up to reflect
- Some Cython files in zmq.core have been split, to reduce the amount of
Cython-compiled code. Much of the body of these files were pure Python, and thus did
not benefit from the increased compile time. This change also aims to ease maintaining
feature parity in other projects, such as
- Context objects can now set default options when they create a socket. These
are set and accessed as attributes to the context. Socket options that do not apply to a
socket (e.g. SUBSCRIBE on non-SUB sockets) will simply be ignored.
- on_recv_stream() has been added, which adds the stream itself as a
second argument to the callback, making it easier to use a single callback on multiple
- A more boolean attribute has been added to the Frame (née
Message) class, so that frames can be identified as terminal without extra queires of
Experimental New Stuff
These features are marked ‘experimental’, which means that their APIs are not
set in stone, and may be removed or changed in incompatible ways in later releases.
- zmq.web added for load-balancing requests in a tornado webapp with zeromq.
remove support for LABEL prefixes. A major feature of libzmq-3.0, the LABEL
prefix, has been removed from libzmq, prior to the first stable libzmq 3.x release.
- The prefix argument to send_multipart() remains, but it continue to behave in
exactly the same way as it always has on 2.1.x, simply prepending message parts.
- recv_multipart() will always return a list, because prefixes are once
again indistinguishable from regular message parts.
add Socket.poll() method, for simple polling of events on a single socket.
no longer require monkeypatching tornado IOLoop. The ioloop.ZMQPoller class
is a poller implementation that matches tornado’s expectations, and pyzmq sockets can
be used with any tornado application just by specifying the use of this poller. The
pyzmq IOLoop implementation now only trivially differs from tornado’s.
It is still recommended to use ioloop.install(), which sets both the zmq and
tornado global IOLoop instances to the same object, but it is no longer necessary.
The most important part of this change is that the IOLoop.READ/WRITE/ERROR
constants now match tornado’s, rather than being mapped directly to the zmq
POLLIN/OUT/ERR. So applications that used the low-level IOLoop.add_handler()
code with POLLIN/OUT/ERR directly (used to work, but was incorrect), rather than
using the IOLoop class constants will no longer work. Fixing these to use the IOLoop
constants should be insensitive to the actual value of the constants.
Add support for libzmq-3.0 LABEL prefixes:
This feature has been removed from libzmq, and thus removed from future pyzmq
send a message with label-prefix with:
send_multipart([b'msg', b'parts'], prefix=[b'label', b'prefix'])
recv_multipart() returns a tuple of (prefix,msg) if a label prefix is detected
ZMQStreams and devices also respect the LABEL prefix
add czmq-style close&term as ctx.destroy(), so that ctx.term()
remains threadsafe and 1:1 with libzmq.
Socket.close() takes optional linger option, for setting linger prior
add zmq_version_info() and
pyzmq_version_info() for getting libzmq and pyzmq versions as
tuples of numbers. This helps with the fact that version string comparison breaks down
once versions get into double-digits.
ioloop changes merged from upstream Tornado 2.1
- added zmq.ssh tools for tunneling socket connections, copied from IPython
- Expanded sockopt support to cover changes in libzmq-4.0 dev.
- Fixed an issue that prevented KeyboardInterrupts from being catchable.
- Added attribute-access for set/getsockopt. Setting/Getting attributes of Sockets
with the names of socket options is mapped to calls of set/getsockopt.
s.hwm = 10
s.identity = b'whoda'
- Terminating a Context closes the sockets it created, matching the behavior in
- ThreadDevices use Context.instance() to create sockets, so they can use
inproc connections to sockets in other threads.
- fixed units error on zmq.select(), where the poll timeout was 1000 times longer
- Add missing DEALER/ROUTER socket type names (currently aliases, to be replacements for XREP/XREQ).
- base libzmq dependency raised to 2.1.4 (first stable release) from 2.1.0.
- bdist for 64b Windows only. This fixed a type mismatch on the ZMQ_FD sockopt
that only affected that platform.
- First version with binary distribution support
- Added instance() method for using a single Context throughout an application
without passing references around.