Regrettable Code, Episode 1

It’s easy to talk about pretty code. Let’s talk about nasty code and see what we can learn from it.

Background

/api/v1/features/team-<id>

The Problem

Let’s Profile!

ncalls  tottime  percall  cumtime  percall filename:lineno(function)...11825    0.049    0.000    0.060    0.000 attrdict.py:26(__getattr__)
384912/432 1.762 0.000 7.012 0.016 attrdict.py:39(deepcopy)
454032/64800 0.650 0.000 6.943 0.000 attrdict.py:49(<genexpr>)
23650 0.023 0.000 0.023 0.000 attrdict.py:58(__setstate__)
46859    0.178    0.000    0.347    0.000 pickle.py:1033(load_dict)
3 0.000 0.000 0.000 0.000 pickle.py:1093(load_global)
3 0.000 0.000 0.000 0.000 pickle.py:1128(find_class)
11825 0.032 0.000 0.088 0.000 pickle.py:1135(load_reduce)
225226 0.497 0.000 0.797 0.000 pickle.py:1156(load_get)
106928 0.180 0.000 0.258 0.000 pickle.py:1170(load_put)
175038 0.393 0.000 0.626 0.000 pickle.py:1199(load_setitem)
11825 0.091 0.000 0.271 0.000 pickle.py:1217(load_build)
58684 0.081 0.000 0.117 0.000 pickle.py:1256(load_mark)
1 0.000 0.000 0.000 0.000 pickle.py:1260(load_stop)
1 0.011 0.011 4.788 4.788 pickle.py:1386(loads)
1 0.000 0.000 0.000 0.000 pickle.py:83(__init__)
1 0.000 0.000 0.000 0.000 pickle.py:835(__init__)
1 1.183 1.183 4.777 4.777 pickle.py:851(load)
58684 0.094 0.000 0.132 0.000 pickle.py:876(marker)
88873 0.230 0.000 0.348 0.000 pickle.py:917(load_int)
2 0.000 0.000 0.000 0.000 pickle.py:967(load_string)
36414 0.116 0.000 0.168 0.000 pickle.py:985(load_unicode)
11825 0.027 0.000 0.056 0.000 pickle.py:999(load_tuple)

Those Still Waters Run Deep

class AttrDict(dict):
"""
Dict that allows attribute access
"""
def __getattr__(self, name):
try:
return self[name]
except KeyError:
return super(AttrDict, self).__getattribute__(name)

def __setattr__(self, name, value):
self[name] = value

@classmethod
def deepcopy(cls, other):
if isinstance(other, basestring):
return other
elif isinstance(other, collections.Sequence):
assert isinstance(other, list)
return [AttrDict.deepcopy(item) for item in other]
elif isinstance(other, collections.Mapping):
assert isinstance(other, (AttrDict, dict))
return AttrDict(
(key, AttrDict.deepcopy(value))
for key, value in other.iteritems()
)
else:
return other
@classmethod
def deepcopy(cls, other):
other_type = type(other)
if other_type in [dict, AttrDict]:
return AttrDict(
(key, AttrDict.deepcopy(value))
for key, value in list(other.items())
)
elif other_type is list:
return [AttrDict.deepcopy(item) for item in other]
else:
return other

This Must Be The Pickle

def _all_features(account_id):
cache_key = _get_cache_key(account_id)
account_features = cache.get(cache_key)
if account_features:
try:
account_features = AttrDict(pickle.loads(zlib.decompress(account_features)))
except (...):
...
else:
return account_features

account_features = _split_features(_query_features(account_id))
cache.set(cache_key, zlib.compress(pickle.dumps(account_features)))
return account_features
In [35]: json_compressed = zlib.compress(json.dumps(account_features))  In [36]: pickle_compressed = zlib.compress(pickle.dumps(account_features))In [37]: len(pickle_compressed) 
Out[37]: 642794
In [38]: len(json_compressed)
Out[38]: 50585
In [41]: start = time.clock()
...: json_compressed = zlib.compress(json.dumps(account_features))
...: print time.clock() - start
...:
0.035571
In [42]: start = time.clock()
...: pickle_compressed = zlib.compress(pickle.dumps(account_features))
...: print time.clock() - start
...:
1.201328
account_features = AttrDict.deepcopy(json.loads(zlib.decompress(account_features)))
json.loads( zlib.decompress(account_features), object_hook=AttrDict)

Except Not Solved :/

Git History, A Few Take-Aways, A Few Random Musings

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store