0.29.0 Released

Saturday, July 06, 2019 | Posted in Release

We are proud to announce the release of Pony version 0.29.0. This release includes several improvements to the standard library’s TCP support. This version contains breaking API changes. We recommend upgrading as soon as possible.

Change how TCPConnection reads data to improve performance

Before Pony 0.29.0, when using the expect call on TCPConnection, a large number of system calls would be generated. expect is used to make it easier to write code to handle framed message protocols in Pony. The prior version would do a system call per frame segment. For example, if two framed messages were received with a corresponding header and payload section were read, four system calls would be made; 2 each for both the header and payload segments. From a performance perspective, this was very wasteful. The performance of the old expect code had gotten worse over the past couple of years as system calls became more time consuming due to various operating system Spectre et al. mitigations.

With the release of Pony 0.29.0, we have switched to creating a large buffer which we populate with a signal system call and then carve off bits as required for proper expect usage. The change has been made for all non-Windows platforms and in our testing, can result in up to a 10x performance boost for high-volume framed TCP protocols.

Make TCPConnection.expect partial

TCPConnection’s expect method is now partial. Previously, you could set the expect value to any size, and the runtime would allocate enough memory to handle. expect was a possible DOS attack vector on Pony applications. Unless the programmer checked every framed message’s payload size against a max allowed buffer size, the program could be made to OOM by providing a sufficiently large number.

As of Pony 0.29.0, when creating a TCPConnection, you can provide a maximum buffer size (or use the provided default). expect cannot be set to larger than that buffer size. Should you attempt to set expect to larger than the buffer size, an error will be generated.

It’s possible that we will end up with other breaking API changes that cascade from this change as it gets used in the wild. There’s already been a discussion of possible changes between Sean and Dipin on the PR for this change.

Alpine Docker release images

Each Pony release now gets Alpine as well as Ubuntu based Docker images generated for it. You can grab them from our DockerHub under the release-alpine tag.

Do not permit leading zeros in JSON numbers

Before this release, the standard library JSON parser would violate RFC8259 by accepting numbers with leading zeros such as “01”, “-012”, “012”. Fellow Brooklynite Luke Rodgers submitted a fix to address the issue. Thanks, Luke.

Fixed

  • Do not permit leading zeros in JSON numbers (PR #3167)
  • Make reading via TCPConnection re-entrant safe (PR #3175)
  • Cleanup TCPConnection GC-safety mechanism for writev buffers (PR #3177)
  • Add SSL tests and fix some SSL related bugs (PR #3174)
  • Fix lib/llvm to support MacOS (PR #3181)
  • Close Denial of Service issue with TCPConnection.expect (PR #3197)
  • Fix return type checking to allow aliasing for non-ephemeral return types. (PR #3201)

Added

  • Allow use of OpenSSL 1.1.1 when building Pony (PR #3156)
  • Add pointer.offset to get arbitrary pointer tags via offset (PR #3177)
  • Add DTrace/SystemTap probes for muted & unmuted events (PR #3196)
  • Add method to AsioEvent to see if an event is a oneshot (PR #3198)

Changed

  • Do not permit leading zeros in JSON numbers (PR #3167)
  • Make TCPConnection yield on writes to not hog cpu (PR #3176)
  • Change how TCPConnection reads data to improve performance (PR #3178)
  • Allow use of runtime TCP connect without ASIO one shot (PR #3171)
  • Simplify buffering in TCPConnection (PR #3185)
  • Allow for fine-grained control of yielding CPU in TCPConnection (PR #3197)
  • Make TCPConnection.expect partial (PR #3197)

AUTHOR

Sean T. Allen

Sean is a member of the Pony core team. His turn-ons include programming languages, distributed computing, Hiwatt amplifiers, and Fender Telecasters. His turn-offs include mayonnaise, stirring yogurt, and sloppy code. He is one of the authors of Storm Applied, and VP of Engineering at Wallaroo Labs.