Using Mongodb for CodeIgniter Logs

Published on May 18th, 2010

Anyone who has ever worked in a production environment knows that your users will find errors you have not even dreamed of. They will find your broken links, they will find your missing files, and yes they will find every database connectivity issue you hoped would never occur. Thankfully we are all great developers and sysadmins so we log everything we possibly can to find and remove these errors.

<p>
    This is great at first but after a while log files become enormous and eat up lots of disk space. The problem
    multiplies when you start getting into a clustered environment (more than 1 server). You now have to read &amp; maintain logs
    on 2+ machines which will most likely lead to neglecting your error fixing duties. Fortunately MongoDB recently hit the
    scene.
</p>

<h2>MongoDB</h2>
<p>
    <a href="http://www.mongodb.org/" target="_blank">MongoDB</a> is a scalable, high-performance, open source,
    document-oriented database. It's ability to accept huge amounts of data very quickly make it perfect for application
    logging. It has built in support for auto-sharding &amp; replication so when your application takes off mongodb will
    scale with you. It uses a JSON based storage system meaning you can give the database an object (from any programming
    language with support) and when you pull it back out of the database it will be the exact same as when you put it in.
    No conversion, no hassle, no bullshit.
</p>

<h2>Capped Collections</h2>
<p>
    Mongo uses things called Collections which are essentially tables in mysql. The main difference is these collections
    are schemaless. I can have 10 fields in one document (row) and 500 in another and they will play nice inside the
    collection. This allows a huge amount of flexibility to the developer.
</p>

<p>
    These collections also have the ability to be "Capped". This means I can set a max document count of 5000 and only the
    most recent 5000 documents will be kept. The older documents are deleted as new ones come in. This is awesome for
    logging because you will never have to rotate your logs in fear that they will become too large and occupy all of your
    file system space.
</p>

Use this command to cap a collection (in a mongo shell):
<blockquote>
    <pre><strong>db.createCollection("my_collection", {capped:true, max:5000})</strong></pre>
</blockquote>
<a title="More Information on Capped Collections" href="http://www.mongodb.org/display/DOCS/Capped+Collections" target="_blank">More
    Information on Capped Collections</a>

<h2>Use in CodeIgniter</h2>
<p>
    By default CodeIgniter writes to log files in /system/logs/ . I have created a class that overwrites that default
    functionality and sends all logs to a mongo db collection. Adding this library to all of your servers will allow all of
    those servers to write to the same mongo collection. I have also added some more data to the logs that is helpful in a
    clustered environment such as the server name, server ip, request uri, and a few more. Just place the MY_Log.php in
    your application/libraries directory and codeigniter should handle the rest.
</p>

<p>
    You can view and fork the class on Github:
    <a href="https://github.com/tomschlick/codeigniter-mongo-logs/" target="_blank">https://github.com/tomschlick/codeigniter-mongo-logs/</a>
</p>

<script src="https://gist.github.com/404697.js?file=MY_Log.php"></script>

<p>
    Feel free to modify data inserted to fit your specific needs. If you find any bugs or have suggestions you can comment
    below or raise an <a href="http://github.com/trs21219/codeigniter-mongo-logs/issues" target="_blank">issue here</a>.