[go: up one dir, main page]

Add `djangae.deferred`

The built-in defer() call in the App Engine SDK suffers a number of issues (some bugs, some design):

  • Deferring a large task (>100K) inside a transaction will not respect the _transactional flag. This is dangerous as your code might work fine for a while, and then things can go very badly wrong if an entity or object grows in size.
  • defer() may or may not use an entity group, depending on the size of the objects. Again, this is a ticking time bomb if you defer a task inside a non-xg transaction; one day your code might just break.
  • defer() uses the built-in Python pickle which is quite limited when there are better options

I plan to write a replacement for defer that has the following changes:

  • It will correctly handle large objects when deferring in transactions.
  • It will use cloudpickle for pickling/unpickling.
  • It will always use an entity group unless a _small_task=True is passed which would never use an entity group (but will throw an exception if you defer something too big).