MongoDB: Convert BSON to JSON

 

MongoDB stores data in the form of JSON-like documents and uses BSON (binary encoded data format) to serialize the documents for storage and data transfers. MongoDB is great at BSON objects, but that’s not convenient all the times. Consider, you are developing a Webb App that requires server side implementation to render JSON objects? Some tiring work involved here..

This post focuses on this aspect of MongoDB and provides a solution with bson module of pymongo with an example (below)


import pymongo
from datetime import datetime
from pymongo import Connection
conn = Connection()
db = conn['myDB']
collection = db['language']
#Creating a collection
db.language.insert({"id": "1", "name": "C", "grade":"Boring", "timestamp":datetime.now()})
#Reading it
print db.language.find_one({"id":"1"})
# Converting to JSON using json module
import json
print json.dumps(db.language.find_one({"id":"1"}))
# Converting to JSON using Tornado
import tornado.escape
tornado.escape.json_encode(db.language.find_one({"id":"1"}))
# Converting to JSON with bson.json_util of pymongo
from bson.json_util import dumps
print dumps(db.language.find_one({"id":"1"}))
#Deleting the collection
db.language.drop()

view raw

mongoJSON.py

hosted with ❤ by GitHub

In this example:

  • We first create a collection, add document to it and read the document
  • We then try to convert the document to JSON format with tornado.escape.json_encode() method or json.dump() and
  • With bson.json_util.dumps() method of pymongo

If you run this program, you would encounter an error like

{u'grade': u'Boring', u'timestamp': datetime.datetime(2012, 9, 10, 2, 41, 13, 437000), u'_id': ObjectId('504db5b91d41c81c1bf072d9'), u'id': u'1', u'name': u'C'}
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2012, 9, 10, 2, 41, 13, 437000) is not JSON serializable

This suggests json.dumps() or tornado.escape module fails to convert timestamps to JSON type.

Solution here is to use dumps method of bson.json_util class to convert a BSON object to JSON..

 

Extra Notes:

bson.json_util class is available in latest pymongo-2.3 distro of pymongo. To uprgae your pymongo instance say, ‘pip install -U pymongo’

ubuntu@ubuntu:~/tornado-2.2$ sudo pip install -U pymongo
Downloading/unpacking pymongo
  Downloading pymongo-2.3.tar.gz (254Kb): 254Kb downloaded
  Running setup.py egg_info for package pymongo
Installing collected packages: pymongo
  Running setup.py install for pymongo
    building 'bson._cbson' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/_cbsonmodule.c -o build/temp.linux-i686-2.6/bson/_cbsonmodule.o
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/time64.c -o build/temp.linux-i686-2.6/bson/time64.o
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/buffer.c -o build/temp.linux-i686-2.6/bson/buffer.o
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/encoding_helpers.c -o build/temp.linux-i686-2.6/bson/encoding_helpers.o
    gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-i686-2.6/bson/_cbsonmodule.o build/temp.linux-i686-2.6/bson/time64.o build/temp.linux-i686-2.6/bson/buffer.o build/temp.linux-i686-2.6/bson/encoding_helpers.o -o build/lib.linux-i686-2.6/bson/_cbson.so
    building 'pymongo._cmessage' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c pymongo/_cmessagemodule.c -o build/temp.linux-i686-2.6/pymongo/_cmessagemodule.o
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/buffer.c -o build/temp.linux-i686-2.6/bson/buffer.o
    gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-i686-2.6/pymongo/_cmessagemodule.o build/temp.linux-i686-2.6/bson/buffer.o -o build/lib.linux-i686-2.6/pymongo/_cmessage.so
Successfully installed pymongo

 

2 thoughts on “MongoDB: Convert BSON to JSON

  1. this will convert:
    id as {“_id”: {“$oid”: “5c26277f2114fc993823e4cf”}
    date as {“$date”: 1546214400000}

    is there a way we get data as stored, like:
    {“_id”: “5c26277f2114fc993823e4cf”}
    {“date” : “2019-01-06T00:00:00Z”}

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.