MongoDB stores data in the form of JSON-like documents and uses BSON (binary encoded data format) to serialize the documents for storage and data transfers. MongoDB is great at BSON objects, but that’s not convenient all the times. Consider, you are developing a Webb App that requires server side implementation to render JSON objects? Some tiring work involved here..
This post focuses on this aspect of MongoDB and provides a solution with bson module of pymongo with an example (below)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pymongo | |
| from datetime import datetime | |
| from pymongo import Connection | |
| conn = Connection() | |
| db = conn['myDB'] | |
| collection = db['language'] | |
| #Creating a collection | |
| db.language.insert({"id": "1", "name": "C", "grade":"Boring", "timestamp":datetime.now()}) | |
| #Reading it | |
| print db.language.find_one({"id":"1"}) | |
| # Converting to JSON using json module | |
| import json | |
| print json.dumps(db.language.find_one({"id":"1"})) | |
| # Converting to JSON using Tornado | |
| import tornado.escape | |
| tornado.escape.json_encode(db.language.find_one({"id":"1"})) | |
| # Converting to JSON with bson.json_util of pymongo | |
| from bson.json_util import dumps | |
| print dumps(db.language.find_one({"id":"1"})) | |
| #Deleting the collection | |
| db.language.drop() |
In this example:
- We first create a collection, add document to it and read the document
- We then try to convert the document to JSON format with tornado.escape.json_encode() method or json.dump() and
- With bson.json_util.dumps() method of pymongo
If you run this program, you would encounter an error like
{u'grade': u'Boring', u'timestamp': datetime.datetime(2012, 9, 10, 2, 41, 13, 437000), u'_id': ObjectId('504db5b91d41c81c1bf072d9'), u'id': u'1', u'name': u'C'}
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2012, 9, 10, 2, 41, 13, 437000) is not JSON serializable
This suggests json.dumps() or tornado.escape module fails to convert timestamps to JSON type.
Solution here is to use dumps method of bson.json_util class to convert a BSON object to JSON..
Extra Notes:
bson.json_util class is available in latest pymongo-2.3 distro of pymongo. To uprgae your pymongo instance say, ‘pip install -U pymongo’
ubuntu@ubuntu:~/tornado-2.2$ sudo pip install -U pymongo Downloading/unpacking pymongo Downloading pymongo-2.3.tar.gz (254Kb): 254Kb downloaded Running setup.py egg_info for package pymongo Installing collected packages: pymongo Running setup.py install for pymongo building 'bson._cbson' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/_cbsonmodule.c -o build/temp.linux-i686-2.6/bson/_cbsonmodule.o gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/time64.c -o build/temp.linux-i686-2.6/bson/time64.o gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/buffer.c -o build/temp.linux-i686-2.6/bson/buffer.o gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/encoding_helpers.c -o build/temp.linux-i686-2.6/bson/encoding_helpers.o gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-i686-2.6/bson/_cbsonmodule.o build/temp.linux-i686-2.6/bson/time64.o build/temp.linux-i686-2.6/bson/buffer.o build/temp.linux-i686-2.6/bson/encoding_helpers.o -o build/lib.linux-i686-2.6/bson/_cbson.so building 'pymongo._cmessage' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c pymongo/_cmessagemodule.c -o build/temp.linux-i686-2.6/pymongo/_cmessagemodule.o gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/buffer.c -o build/temp.linux-i686-2.6/bson/buffer.o gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-i686-2.6/pymongo/_cmessagemodule.o build/temp.linux-i686-2.6/bson/buffer.o -o build/lib.linux-i686-2.6/pymongo/_cmessage.so Successfully installed pymongo
Thanks a lot pal! I was looking for this from quite sometime!! U awesome! 🙂
this will convert:
id as {“_id”: {“$oid”: “5c26277f2114fc993823e4cf”}
date as {“$date”: 1546214400000}
is there a way we get data as stored, like:
{“_id”: “5c26277f2114fc993823e4cf”}
{“date” : “2019-01-06T00:00:00Z”}