cat /dev/brain

github3.py

After finishing finals, I started working on my project to wrap the Github API. Recently, I came across the area of the API dealing with creating a download on a repository. The one thing that has made this project so enjoyable has been requests. Upon reaching this section, I noticed that the order of the data mattered for the form when sending the POST request to S3. The problem is that dictionaries in python do not preserve the order, e.g., the order in which you add elements does not mirror the order you may get them out.

data = [('key', 'downloads/github3.py/foo.txt'),
    ('acl', '...'),
    ('success_action_status', '...'),
    # ...
    ('file', open('foo.txt', 'rb').read())]
data_dict = dict(data)

# Do some other stuff to set up the request

for (key, val) in data_dict.items():
    print('{0} => {1}'.format(key, val))

# Prints:
# success_action_status => 201
# file => <contents of file>
# ...
# key => downloads/github3.py/foo.txt
# ...

Knowing this, I attempted to make an OrderedDict class that would mirror the functionality of Python 2.7's OrderedDict class to keep compatibility. So I put together the following rather quickly

class OrderedDict(dict):
    def __init__(self, items=[]):
        super(OrderedDict, self).__init__(items)
        self.__items__ = list(items)

    def __repr__(self):
        return 'OrderedDict({0})'.format(str(self.__items__))

    def __setitem__(self, x, y):
        super(OrderedDict, self).__setitem__(x, y)
        self.__items__.append((x, y))

    def items(self):
        return self.__items__

It's a simple workaround for Python 2 users and it worked. This still caused problems with Amazon's S3 API. I ended up tracking the problem back to a function in requests called _encode_params. The function turns any data provided into a regular dictionary so it loses its ordering.

The Way Forward

I found a recipe to create the multipart/form-data, ensuring that I was constructing the data for the request correctly. The problem I'm facing now seems to be that the connection is timing out on Amazon's end. I'm not certain why this is happening but there are a few possible explanations which are my fault. As soon as I figure out what I have done wrong (assuming that it is my fault), I will write a dissection of issue.