Fast Networking with Python

• Itamar Shtull-Trauring
• http://itamarst.org

What do we mean by 'Fast'?

• I/O bound
• CPU bound

I/O bound

• Not enough bandwidth
• High latency
• Solution: improve protocol

CPU bound

• Can't keep up with network
• Networking code is slow, or
• Business/application logic is slow
• Example: batch processing

Our goal

• Network code fast enough
• I/O bound

Lazy Optimization

• Upgrade to Python 2.4
• Buy new hardware

Algorithms

• Python can be faster than C

Example: Scheduling

• while True:
      for t in getScheduledEvents():
          t.run()
      sleep(getTimeUntilNextEvent())
  

Example: Scheduling

• glib: iterate over all timers O(n)
• Twisted: heapq of timers, only get newest O(k)
• Twisted is faster for large number of timers

Data Copying

• Python strings are immutable
• Adding two strings copies data

Bad String Concatenation

• s = ""
  for i in chunksOfData:
      s += i
  

Do This Instead

• l = []
  for i in chunksOfData:
      l.append(i)
  s = "".join(l)
  

Comparision

Lower-level Optimization

• Reduce allocation
• Reduce memory copying

buffer() Is Your Friend (Kinda)

• buffer() is like char*
• Non-copying reference to substring

buffer() vs. string copy

Writing without buffer()

• def doWrite(self):
      amountWritten = write(self.data)
      if amountWritten:
          self.data = self.data[amountWritten:]
  

Writing with buffer()

• def doWrite(self):
      slice = buffer(self.data, self.offset,
                     len(self.data)-self.offset)
      amountWritten = write(slice)
      self.offset += amountWritten
      if self.offset == len(self.data):
          self.data = ""; self.offset = 0
  

Improving Python

• Control over memory allocation
• Better buffer type
• Changes to existing built-ins

Buffer API

• C API for getting at char*
• Implemented by strings, buffer(), mmap
• Supported by many APIs, e.g. file.write()
• Needs to be supported more

• >>> s = "10lala"; b = buffer(s, 0, 2)
  >>> str(b)
      '10'
  >>> int(b)
  ValueError: invalid literal for int(): 10lala
  

array.array

• Can be used as mutable array of bytes
• Can't do non-copy slices
• Missing .split(), .find(), etc.

Example: Non-copying split()

• str.split() copies into new strings
• Modified C code that returns buffer()
• Non-copying version is 8% faster
• No improvement for small strings

Sockets and Files

• file.read(), socket.recv() allocate on read
• file.readinto() - unsupported
• socket.recvinto() - nonexistent

Other Options

• C/Pyrex/C++ extensions
• Fusion: C++ protocols for Twisted (http://itamarst.org/software/
• Flumotion technique (Twisted + glib C code)