Jump to content

Issues writing DTI data to file


Recommended Posts

Hi there,


I am having issues writing data from a DTI table to file. I created ladder logic to write data to files on a regular basis. I couldn’t use the normal Data Sampling functions, as although I need to log to file regularly, there are times when the (measured) system state is unknown. At these times I have to stop regularly logging to file and wait until a certain period has elapsed before I can accurately assess the system state and start logging to file again.


So what I do instead is regularly write a row of data to a DTI table, inserting it at the top so that the rows all move down one, using the Insert Row in DTI function. When the system state is known, I write a row of data from the DTI table to file every 2 seconds. When it is unknown, I record the row of data in the DTI file, then when the system state becomes known again, I "catch up" writing all of the rows I have not yet written to file from the table, one at a time. This has to be done one row at a time as the files are appended to one row at a time and so the oldest data is at the top of the file, but at the bottom of the DTI table.


I am logging rows one at a time using the Store DTI to File function, appending to file and using the CSV option to make sure I get a CSV file as well as the default UDTF file. This is because it’s easier for the end user to open a CSV file rather than have to convert it first, even if it does come inside a .zip file. I was sent a converter that converts UTRF and USMP files to CSV, but not UDTF files - is a converter available?


When logging to file, I start at a certain time and use that as a date/time stamp (in the format yyyy-mm-dd_hh'mm'ss) for my file name, so before the weekend, I set the PLC logging every 2 seconds, and it started with a file name of "Data_2014-12-12_18'24'37".


I set the ladder up so that if the number of samples in the file exceeds 60,000, then the file name changes, so that I can start logging to a different file and so split up the data into multiple smaller files that will open, even in old version of Excel.


So after a couple of days, it made a new file name, "Data_2014-12-14_03'46'06" and started logging data into that.


The issue is that while the PLC made both a UDTF file (which I can’t open) and CSV file (in a .csv.zip file, which I can open) for the first file ("Data_2014-12-12_18'24'37.udtf" and "Data_2014-12-12_18'24'37.csv.zip"), it didn’t make a CSV file for the second file.


I have looked for these files on the SD card of the PLC using both Uniapps and FileZilla, via FTP.


There is more (perhaps linked) strange behaviour when looking at the SD card contents via FileZilla/FTP, and pressing F5 regularly to refresh the view of the contents of the DT folder on the SD card.


There is, in the DT folder, a .csv file, with the first file name ("Data_2014-12-12_18'24'37.csv"). This file is not inside a .zip file, and, even though the logging to file is now stopped, the file is seen to grow in file size from very small to about 7.5 MB, then another strangely named file, like "ziFheLDg", or "ziMqNiMw" will appear, then that file will disappear and the .csv file will shrink in size and then grow again.


This seems like it might be the PLC regularly re-writing the UDTF file to a CSV file, then zipping it into a .csv.zip file, but that’s just a guess. But the thing is that it’s still doing this for a file that’s already been written to, and so doesn’t need re-writing to a .zip file.


When the file name has been changed and a new file is being written to, how can I get the "also save as CSV" option to stop writing the old file to a .csv.zip file and start doing it on the new file? 

Link to comment
Share on other sites

When writing the DT file to the SD card is it possible that the create CSV bit is reset when the name is changed? This should be the only cause of the file not being created.


I was not unable to re-create this issue by changing the file name alone.


Also, out of curiosity why are you writing a single row to the SD card once at a time? It is possible to write any number of rows or an entire table at once. I understand why you may only log rows one at a time to the DT, but all of the rows can be immediately written to the SD file when necessary.

Link to comment
Share on other sites

Thank you for the suggestion. I don't think that's possible in this case, as I had the bit set with a constant ("#1"), so it was always set when an individual row was written to file. Unless there's still some way it could become unset?


As I mentioned the reason why I was not writing multiple rows was because I was using the Insert Row in DTI function to fill my DTI table, as that was a good way to do it, but then you end up with oldest data at the bottom and newest data at the top, as opposed to the file, which is filled from top top bottom (appending data to previous data). So if you wrote a block of rows from the DTI, then it'd be in reverse order compared to the rows above and below that block.


I need to write the rows to file fairly frequently, as the user might stop the PLC at any time and expect all of the data to be saved on there, so I save rows of data as they are sampled/calculated, unless the system is in the "unknown" state I described.

Link to comment
Share on other sites

As an update, I carried on playing with this, and when I deleted the "Data_2014-12-12_18'24'37.udtf" file to see what that would do, the writing to CSV file stopped altogether. It even stopped for the Data Sampling function that I had set up in the same project, when I turned that on and off a few times. Updating the program on the PLC didn't work and a few resets did little either. Some time later, though, it seemed to come back and start writing a csv.zip file, again with the odd cyclic filesize-changing .csv file. but it was not making a CSV file (in a ZIP file) of the most recently saved file, but the one just after the deleted file, (i.e. the "Data_2014-12-14_03'46'06" file), not the other files that I saved to after that one had been created.


I have saved many files since, turning this logging on and off, and it's still going around and around, cyclically creating a "Data_2014-12-14_03'46'06.csv" file (presumably saving it into the "Data_2014-12-14_03'46'06.csv.zip" file).


I think I might have to delete all of the files and see what that makes it do... So many of these actions are totally opaque and there's very little to tell me what the PLC's actually doing.

Link to comment
Share on other sites

This does appear to be a strange issue. Would it be possible to send us your program for further review? You can send it to support@unitronics.com.


Also, can you elaborate on what you mean by cyclic filesize changing? Are there other function blocks that are attempting to save the data table information to the SD card using the starting file name?


Which version of UniLogic, and UniStream firmware are you currently using? If you have not updated to the most recent version I would recommend doing so. The most recent relase of UniLogic is 1.9.19 and the UniStream firmware is 1.9.6.


Let us know if completely removing all of the files from the SD card solves any of these issues.

Link to comment
Share on other sites

Yes, I've updated the software and firmware recently and I'm on UniLogic 1.9.19, plus the Firmware Manager says the UniStream firmware is 1.9.6.


The cyclic file size changing thing I was mentioning was that when you look at the files in the DT folder on the PLC (I was using FileZilla over the Ethernet connection), and update the view of these files regularly (pressing F5 ever second or so), there was a .csv file, with the same name as the file name that was being logged to, (e.g. "Data_2014-12-12_18'24'37.csv").


This file was not inside a .zip file, and, even though after the logging to that file was stopped, the file was seen (when refreshing FileZilla’s view of the files in the DT folder)  to grow in file size from very small to about 7.5 MB, then another strangely named file, like "ziFheLDg", or "ziMqNiMw" would appear. Then that file would disappear and the .csv file would return back to a small size. That would keep happening, and the size of the .csv file would grow again, in a repeating cycle.


As I mentioned before, it seemed like it might be the PLC regularly re-writing the UDTF file to a CSV file, then zipping it into a .csv.zip file (I've noticed when zipping things on my own PC that you get intermediate files as a zip process is in progress, when the zipping process is complete and the zip file has been written.


But this behaviour was continuing after the logging had stopped, so the .zip file didn't really need re-writing. But that’s just a guess of what it might be.


Other contributing factors might be that I was asking the PLC to do a lot in each ladder cycle (or at least some of them). I was working with large DTI tables, and have seen several watchdog timer issues, where the PLC has stopped, and so I've put a stopwatch in the ladder to tell me how long each ladder cycle takes (the time it takes between each return to the end of the last function in the ladder, using the stated stopwatch resolution units 64/66,500,000 seconds). While some cycles are relatively short, at about 3.5 ms, whenever a new row is added to the top of the large table (8640 rows, with 2 columns of strings, 12 columns of reals and 3 columns of bit data), the ladder cycle time reaches about 225 ms. This was a pared back table, as previously this was causing a lot of PLC CPU stops.


It’s a shame that there’s no alert when this happens so that the user can be told to restart the PLC – as in some ways it still looks like things are working (like you can still move through the screens, use some on-screen controls, etc.), but the CPU has stopped in the background


I have tried leaving the PLC on for a long time with the logging all turned off, and deleting all of the old data (.udtf and .zip files) from the DT folder, and the PLC is not showing the same signs of the cyclic file size changing .csv file or the appearance of the oddly named files. So either leaving it for long enough to sort itself out or deleting the old files off the SD card seems to have helped.


If I update the view in FileZilla very quickly, I sometimes see the .csv file now, but it doesn’t seem to be so prevalent or permanent as it was, even when the PLC is pushed to log to file very frequently. 


I think pushing the PLC to do so much in some ladder cycles may have affected the reliability of the system and its ability to definitely write data to file(s) at precise and regular intervals, anyway. It might also have contributed to seeing things (like the .csv file cyclically changing in size, etc.) that might otherwise happen so quickly in the PLC’s ladder cycle "downtime" that they don’t normally get seen.


I am going to back off the tasks that I'm asking it to do, as this reliability of saving the data to file is key.

Link to comment
Share on other sites

The issues could very likely be caused by the frequency of the information being written to the SD card.


I would try writing this data table information to the SD card less frequently. This can be accomplished by writing the entire table only after a certain number of rows or after a longer period of time.


Hopefully this should resolve this issue.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...