Datalog Practices

Before beginning a data logging project it's important to ask a few simple questions about the nature of thing you are logging. Here are a few sample questions:

• What am I sensing and how often does it change?
• Will the values change rapidly or slowly?
• How large is the difference between the maximum and minimum readings I might see?

Once you have a sense of the behavior of the thing your are logging, figure out what you want to use the data for:

• Are you logging to look at trends over a long period of time or are you trying to use the information in real-time?
• How much precision (both in the actual values and the timing) do you need?

The final major issue to consider is how/where you are storing the data and how you want to be able to access it:

• Do I want to post to the internet (a mysql database) or is this on my skateboard so I'll be storing it on a flash card?
• How do I want to retrieve the data and how often do I want to be able to do that (anytime, once a week, whenever I have a chance).
• How much memory space do I have and how frequent are my readings and how big are they?

That last point is important. If you are taking a reading once a second and that reading has a time stamp and value you might be looking at 100 bytes per reading, which means that in 1 day you'll have generated: 100bytes *60sec * 60 min * 24 hours = about 8.5 megabytes of data.

You may not have the space for that, or depending on what the data you are looking for you might not have to store every reading.

For example if you are trying to track when lights go off or on in a room with a photosensor, you can design your system so that it only posts a reading when it notices a big change (up or down) from the previous reading. Even if this system were sampling every tenth of second, you would generate only a small but relevant set of data.

The overall point is to think about the nature of what you are tracking, what you want to do with that datastream and how much space and computing power you have.