How to Store Data Logger Data to a Memory Card

You may have identified the need to add external data storage to your data logger with flash-memory cards. But do you know which CRBasic instruction to use to do it? In this article, I’ll help you choose the most suitable instruction for your application. In a future article, I’ll discuss some more complex issues and offer handy card tips and tricks that you may not be familiar with.

If you're not that familiar with memory cards, you may find it helpful to review the "Pick a Memory Card, but Not Just Any Memory Card" blog article.

How do I store data to a memory card?

There are two CRBasic instructions that are commonly used to instruct the data logger to write final-storage data to a memory card:

1. The CardOut() instruction
2. The TableFile() with Option 64 instruction

These instructions are added between the DataTable() and EndTable() instructions as shown in the examples later in this article. Note that you cannot use both CardOut() and TableFile() Option 64 in the same data table. You must choose which instruction is best for you. So, why are there two instructions to store data to a card? Is one method better than the other?

Before answering these questions, it is important to understand that the maximum size of any one file stored on a data logger is 2 GB. This is because the data logger natively supports signed-4-byte integers. This data type can represent a number as large as 2³¹, or in terms of bytes, roughly 2 GB. Hence, 2 GB is the maximum file length that can be represented in the data logger directory table, and consequently, that is its maximum file size.

Partially because of this file size limitation, and also due to the high cost of large memory cards, Campbell Scientific data loggers did not support memory cards larger than 2 GB prior to operating system (OS) 25 in the CR1000 or CR3000. However, as demands for high-frequency data and greater data storage grew, support for cards up to 16 GB was added to the data logger operating system.

The support for cards larger than 2 GB presented a conundrum. How can we write more than 2 GB of data to a card if the maximum size of a final-storage file is 2 GB? The solution was provided through the addition of a new option (Option 64) to the TableFile() instruction that allows more than one final-storage data file to be created from the same data table. Hence, many small files can be created from the same data table, thus allowing up to 16 GB of final-data storage from one data table.

So, prior to OS 25, only the CardOut() instruction was available to store data to a card, and only one file could be created for each data table. However, with OS 25 and later, TableFile() Option 64 is also available for storing more than one file for a given data table to a card.

Is one method of writing data to a card better than the other?

An advantage of the CardOut() instruction is that it is very simple to add to your program. More information on the CardOut() instruction is available in the CRBasic Editor Help files.

In many applications, however, the TableFile() instruction with Option 64 has several advantages over the CardOut() instruction. These advantages include:

• Allowing multiple small files to be written from the same data table so that storage for a single table can exceed 2 GB. The TableFile() instruction controls the size of its output files through the NumRecs, TimeIntoInterval, and Interval parameters.
• Faster compile times when small file sizes are specified
• Easy retrieval of closed files via File Control utility, FTP, or email
• Closed files are safe files. Anytime a file is open for writing, it can become corrupted if a power loss occurs or if the writing is interrupted for any reason. TableFile() Option 64 will close files at the programmed time or record interval. Once those files are closed, they are safe from this type of corruption.

CardOut() Program Example

Public batt_volt
DataTable (Test,1,1000)
	DataInterval (0,15,Sec,10)
	CardOut (0,-1)
	Sample (1,batt_volt,FP2)
EndTable
BeginProg
	Scan (1,Sec,0,0)
		Battery (batt_volt)
		CallTable Test
	NextScan
EndProg

TableFile() Program Example

Public batt_volt, OutStat As Boolean, LastFileName As String * 32
DataTable (Test,1,1000)
	DataInterval (0,15,Sec,10)
	TableFile ("CRD:Test_",64,10,0,1,Min,OutStat,LastFileName)
	Sample (1,batt_volt,FP2)
EndTable
BeginProg
	Scan (1,Sec,0,0)
		Battery (batt_volt)
		CallTable Test
	NextScan
EndProg

Data Collection and Access

CardOut() files are open for writing as long as the CardOut() instruction is running and are, thus, always available for remote data collection. With TableFile() Option 64, the most recent file is open and available for remote collection. After a file is closed, data can be accessed only by first retrieving the file (for example, via the File Control Utility, FTP, email or, copying files from the card).

For both CardOut() and TableFile() Option 64 instructions, open files on the card become an extension of the table memory. Any time data is retrieved via the Collect or Custom Collect utilities of LoggerNet, PC400, or RTDAQ, the internal SRAM is searched first, then the card. If a file is found on the card with the same name and file header as the data table in the internal memory, the data from the card is appended to that file.

Note: To be considered identical, the file must have been created on the same data logger using the same program.

You can view your memory card data using data logger support software, such as the View Pro client application in the LoggerNet software package. You can also access your memory card data through the data logger program using data table access syntax such as TableName.FieldName. (Refer to the CRBasic Editor Help file.)

CardConvert Software

Note that for both CardOut() and TableFile() Option 64, data is stored on the card in TOB3 format. TOB3 is a binary format that incorporates features to improve reliability of the CF cards. TOB3 format is different from the data file formats created when data are collected via a communications link, which is TOA5 or ASCII format. Hence, data files that are read directly from the CF card need to be converted into another format to be human readable. You can convert files from TOB3 to TOA5 or other formats using the CardConvert software that is included in LoggerNet, PC400, and PC200W.

Conclusion

I hope this article has helped you to understand how easy it is to add a CardOut() or TableFile() instruction to a CRBasic program that will direct your data logger to store data to a memory card. (For details on each of these instructions, see the CRBasic Editor Help files.) In a future article, I’ll address some more complex card issues and offer handy card tips and tricks.

If you have any questions or comments about memory cards, post them below.