Problem with V1210 - PLC goes to stop mode

[small7]

Hello 
I have problem with Vision V1210. Every few days (1 or 2 week) PLC goes to stop mode, screen is black (i don't have any error code) and i can't do anything with it. Only restart by power off helps. After restart,  PLC remember alarms which was occurred In the time when PLC does not responding ( it's like program in PLC was work but it not work properly).  I have a new version of Visilogic and new OS is installed on PLC. I replaced PLC to the new unit. But this is still happening.  How i can diagnosis what is wrong. How can I read what errors occurred.

Thanks for help

[Alexander]

Topic moved to Vision PLC and VisiLogic section.

 

When this issue occurs can you still communicate with the unit? Is it possible that the system integer controlling the brightness, SI 9, has been set to 0 within the program?

 

Also, when the screen is displaying black can you access Info Mode by pressing and holding on the screen for 5-7 seconds? This will automatically set the brightness higher so the HMI screen will be visible. Then you can navigate to operands and modify the SI value manually. If this works it is only a temporary solution if this works as the application will likely reset this value back to 0 at some point.

[small7]

No i can't communicate with PLC and can't access  to Info Mode. When PLC does not responding, program work not properly. PLC remember alarms but the rest of the program work not properly.

[Alexander]

What type of I/O is this controller using. Does it have snap in or local I/O attached? If the I/O draws more current than the power supply is able to provide it could trigger the controller to lose power.

 

When this occurs I would recommend checking the I/O modules to see if they are running and check the power to the V1210 to see if the voltage has dropped below 24VDC. If possible also check to see if the current has dropped as well. It could be that when a combination of I/O or drives on a single circuit are turned on it draws too much power and causes a units to lose power.

[small7]

This is large system, Vision have snap V200-18-E46B and more I/O like EX-D16A3-R08, 4 x IO-D16A3-R08, 2 x IO-ATC/AI8 and IO-AO6. I'll try check power supply  current and voltage today., but it will be with working unit.  Vision from two days (after restart) work properly. Today I am going to a place when PLC is working, can I check anything more maybe some SI or other's system registry? 
If there is problem (like pure voltage or anything else) why Vision don't report any error. It could be helpful with diagnosis problem.

 

[small7]

I checked the voltage - 27 V and current -  0,8A, The power source has an current output 6A, so this is ok. i have tried subrutine "_RUN_TO_STOP" but it doesn't work ( it is like PLC was work so subrutine don't reset PLC). I changed  value SI9 to 20  in "screen saver". Maybe there is error code  when PLC not work properly, but i cant't see it because screen is black (SI 0 set by subrutine screen saver). 

[Alexander]

Unfortunately there are not any system registers or bits that will indicate any issues with power. It appears as if the unit is powering off or experiencing a surge which can not be anticipated by the PLC. Do you notice that this occurs during any specific times (when an output or drive turns on or off)?

 

When you checked the voltage and current was this when the unit was displaying the black screen? When displaying the black screen is the I/O units still receiving power?

 

I would recommend checking the current draw directly from the power supply if possible. The voltage is slightly high since the V1210 runs on either 12 or 24VDC. The V1210 itself draws a maximum of 1 amp at 12V and 0.5 amp at 24V. Where did you measure 0.8 amps from? Even though the power supply is rated for 6A it may be supplying less or experiencing periodic surges.

 

Also, does this occur with any other similar system or is this isolated to this individual unit?

[small7]

"Do you notice that this occurs during any specific times (when an output or drive turns on or off)?" 
I don't know because i'm told about it a few hours after when it happened. This unit is instaled in water treatment station, So when water reaches low level limit then others device like pumps was stopped and personel knew about problem with PLC, but like i said problem started a few hours before. During this few hours Plc notice alarms (like low limit level of water) but don't execute rest of program like start subpomps to produce water. 

"When you checked the voltage and current was this when the unit was displaying the black screen? When displaying the black screen is the I/O units still receiving power?"
No, i checked it when PLC run properly, i must wait when it will be happen. I was changed SI9 to 20 in "screen saver" maybe error code will be  displayed on the screen . 

​

"The voltage is slightly high since the V1210 runs on either 12 or 24VDC. "
In Technical Specifications of V1210 is writed that permissible range of voltage is 10.2-28.8VDC. By the way this type of  power source is often used in similar systems and i don't have any problems. I measured Voltage on power source and on Vision, current only on output of power source. 

"Also, does this occur with any other similar system or is this isolated to this individual unit?"

One time i have problem with similar system but i finded a problem. It was broken earth screen of level probe that was causing a short circuit between grounded earth and 0VDC.

After replacing level probe the problem disappeared. In this system i checked it. There was the same problem with radio modem Satel (there was factory-made ground to 0VDC of modem). I isolated the power of modem from Vision and i changed communication, between Vision and modem, from RS 232 to RS 485.

[Alexander]

I would recommend checking the power when it occurs and if possible have the technicians report if they notice any combination of devices being turned on/off that triggers the condition.

 

If the program is not executing once the screen has gone dark then it is likely that the controller is powered off. It is strange that the unit boots up and works properly up after cycling power. Usually as long as the unit is receiving enough voltage and current then it will stay powered or even cycle power once the requirements (24VDC & up to 0.5 amps) is restored. Having to cycle power indicates either hardware or power flow being the issue if you can no longer communicate with the unit.

 

If you continue to experience issues please send an email to support@unitronics.com.

[small7]

"Having to cycle power indicates either hardware or power flow being the issue if you can no longer communicate with the unit"
I'll try communicate with the unit when it occurs. If you're talking about the hardware, it is already exchanged for a new device (V1210) but it still happens. Mayby problem is with I/O?

[Alexander]

If it also occurs with a replacement module it is likely the environment that it is installed in (electrical noise, or power surges, failures) that is causing the issue. As you mentioned this could be from certain I/O points being turned on/off.

[small7]

But I changed only PLC maybe there is a problem with Snap or I/O modules.

[Martin1]

Are you using remote CanBus modules. If so, check the earthing. Make sure it is only earthed on one side. I had this issue before

[Alexander]

Yes, It could be caused by the I/O or attached modules or from the environment that all of the equipment is installed in.

 

Is it possible to disconnect the V1210 and all of the I/O and set it up in an office environment without any of the I/O points connected? If the issue continues to occur it could be the hardware, if the issue is resolved it may be the environment or a combination or the I/O with certain points being on/off.

[small7]

Are you using remote CanBus modules. If so, check the earthing. Make sure it is only earthed on one side. I had this issue before

No i'm not using Can bus at all. I'm using communication by RS485 but I earthed it only on one side. 

[small7]

Yes, It could be caused by the I/O or attached modules or from the environment that all of the equipment is installed in.

 

Is it possible to disconnect the V1210 and all of the I/O and set it up in an office environment without any of the I/O points connected? If the issue continues to occur it could be the hardware, if the issue is resolved it may be the environment or a combination or the I/O with certain points being on/off.

No it isn't possible to disconnect all unit's, It most work all the time because this system produces water for the people. To do this i would have to replace all units.

[Alexander]

Since it is not possible to test when disconnecting the units, you will have to test if the issue occurs again.

 

When/if this happens monitor the input/current to the PLC and each I/O point. Also, record which inputs/outputs are on or off at this point since it may be a combination of points that causes this issue. This information may help to determine the root cause of the problem.

[small7]

I have new information. In tuesday PLC goes to stop and on the screen was error like on the image 

[Alexander]

This type of error is caused by a fault within the logic. An unlikely sequence of events or a faulty FB within the program could potentially cause this issue.

 

To review the code within VisiLogic you can use the Ldr code to search for the subroutine that causes the issue. This feature can be found under the Edit menu, then selecting Find Subroutine by Ladder Error Code.

 

This error may also be generated by a fault in the O/S. I would recommend either updating or reinstalling the operating system within the controller.

[Walkerok]

I have watched this post for a couple of weeks hoping magic would strike because I have multiple V570 units that behave very much in the same way. I have a single customer with 100+ V570 units. The mix is approximately 49 units installed in a common cabinet with a VFD running the motor application. Exactly 51 of the units are on soft starts running their motor applications. All PLCs are the same exact hardware, same program, same power supplies, but VFD installed units are ALL ok and I have 11 soft start units stopping at different intervals for unknown reasons. (stopping is not stopping by the way it is the PLC frozen with whatever it was doing last and is only correctable by power cycling)

Facts:

All 100 units:

V570-57-T20B

V200-18-E6B

Installed outside in either NEMA 3R or NEMA 4/12 enclosures

About 20 of the units were created when Visilogic was before version 9.7.9 was in effect

The rest are either version 9.7.9 or 9.7.24

Now those of you that work with the Unitronics brand a lot may have noticed that version 9.7.9 had (has) some serious bugs, however 9.7.24 seems to be pretty solid with all defects from 9.7.9 correct (or fixed enough I can't find anything particularly wrong)

So let’s go through the rest of it. What are the major differences between a soft start and a VFD panel. The obvious difference is noise, however all of the VFD units are fine and VFDs are dramatically more noisy than a soft start. How about the installation? I had a tech that works for me go to 6 sites and at the first site the installation looks like it was done by 3 year olds. So aha we think its just bad installations. However, after the other 5 sites are visited they are text book installations with good grounding, no shortcuts taken on shielded cable, clean connections with no stranded wire flaring out......NEC would be proud.

What is the other major difference between a VFD panel and a soft start panel.....heat!

VFDs make heat and soft starts do not. So the big difference between the multiple installations is that the VFD the boxes are almost always warmer than ambient temperatures and soft start boxes are not. This leads me to what I believe could be the cause of all the problems and that is not noise but is condensation. If you look a dew point charts relative to humidity it does not take cold temperatures to create condensation, but just a large enough temperature differential (at low enough overall temperatures) between the outside ambient temperature and the internal box temperature.  I should mention that all of my soft start panels are installed in NEMA 4/12 enclosures with no venting or cooling fans (soft starts don’t need them) all of the VFD panels have cooling fan systems that allow outside air to be used to cool off the heat created from the VFDs.

 

So I don’t have a final answer yet, but we have installed different solutions on 4 of the affected panels.

1. Installed AC and DC noise suppression on all outputs and a heater to the enclosure
2. Installed a 120VAC noise filter just ahead of the DC power supply and a heater
3. Installed both AC and DC surge suppression , 120VAC noise suppression and a heater to the enclosure
4. Installed only a heater and nothing else.

 

I will report back after I hopefully have a conclusion, but my gut is telling me that condensation is the cause and not noise (Unitronics does not formalin coat anything), however, since I am not in the business of guessing I decided to alter 4 sites with different solution possibilities to the multiple test sites to come to what will be the real answer.  I have been pressed to give a solution before I actually find an answer so I will be installing  external watchdog timers on the remaining 45 units by programming an output of the PLC to do nothing but pulse back and forth every 5 seconds.  If the PLC freezes in either position the watchdog will reset the 24 VDC power to the PLC.

 

Please do not respond to this post by asking me if I have replaced hardware, firmware, shielding, VFD carrier frequencies…….I have literally done everything that I know can be done at least twice over the last 7 months.  I started out thinking this was a hardware problem, then a firmware problem, grounding problem, shielding problem and nothing has worked.  The only thing I had not tried is adding heat to the enclosures so that is where I am at now with the 4 units.  If anyone thinks of something other than what I have mentioned here please post.

 

Wish me luck …I need it

 

Keith

 

[Flex727]

This is engineering to make engineers proud. Your plan looks great! Keith, I hope you will report back your results.

Also, can you provide some specifics on the hardware for your external watchdog timer?

[Walkerok]

Well the results are in and I was not correct on the condensation hypothesis.  The final end customer has placed a chemical injection pump in tandem with one of the expected devices and there was no noise canceling hardware installed on it.  Even worse the output they decided to piggy back off of changes state once every 30 - 120 seconds depending on conditions.  How do I know this?

 

Using the capabilities the Enhanced vision products have I put a data logging .CSV write to the SD card function that took the status of every input, output, analog input, analog output, alarm, and system messages every 1 second.  I then had a the PLC create a new file every day at midnight. 

 

Just to follow a complete process.  The two units without any noise canceling hardware on outside relays, starters, and valve coils failed at least once per day.  (Look at the previous post to see the 4 configurations tested)  Over the same time period the two units with AC and DC noise mitigation hardware installed ran 24 hours a day with no problems.  After 3 days of failures the noise canceling parts were installed and all 4 tested units then had AC and DC noise devices.  After installation of the noise cancelling hardware all 4 sites have run flawlessly.  By logging this way I was able to narrow down my focus to just a single output from the PLC and further determine that the contactor to run the chemical pump was the cause of the noise.  I was also able to confirm that the PLC was absolutely, definitely, positively, without any question or ambiguity ceasing processor function and staying stuck on whatever was being executed at the time of the brain freeze until a loss of power allowed a system reset.

 

The 120 VAC noise mitigation parts are a combination of a capacitor and resistor OKAYA product number XAB1201

The 24VDC noise mitigation (transient voltage suppressor) parts are a Bi-directional zener diode at 28 volts.  These are part number SA28CACT made by Littlefuse.

 

The good new is the fixes are not expensive and we found them.  The bad news.....the end customer changed something then swore they never touched anything and cause a whirlwind of pain and anguish that we have to pay for as a company.  Oh well welcome to industrial electronics.

 

For FLEX727

On the watchdogs I had to have installed it is an ugly adder but works well.  There is a company AIROTRONICS that has two watchdog timers (one with excitation at power up and one with excitation only when the first pulse is received).  I thought I could the model with excitation on power up and it works but only once (this is part number TGLR )  The other product is what actually worked for me and is part number TGMPL(several more selectable configurations but search this number in Google and it will come up).  To make this work I had to run the first signal pulse through the normally closed contact on the relay.  The relay worked but a little too well.  There is not a long enough time delay to power down the PLC before the power is re-engaged to create a PLC reset.  I had to add a delay on make timer along with this to actually cut off power long enough to create a PLC reset. (tried to find a usable reset on the PLC CPU and there is not one that mere mortals can access).  There is a rest on the CPU as one would expect but soldering wires to the surface mount chips was way beyond my mandate and beyond anything I could ask a field guy to install. I will be talking to the timer maker to see if they can add a time delay on the reset function and then it will be a clean installation.

 

For the creators

Why does the internal watchdog built into the CPU of the V570 not ever figure out the CPU is locked and reset the CPU.  looking at the wiring diagram for the main processor it has the separate watchdog circuitry I would expect it to have.  Is this something that has been accidentally changed in a past firmware update and forgotten ever since? Is it an intentional choice....if so please explain?  I am not trying to shoot arrows, I love my Unitronics-products and they have worked well for many applications for years, but this is a serious condition that should be investigated. It is an easy fix in firmware (at least it appears to be) it just needs to be done. (or redone as it were)

 

Thank You

Keith

[sgull]

Hi Keith

 

Thanks for your post, It will definately help us in future troubleshooting.

The above post should be PINNED by unitronics as a reference to the improtance of noise suppression.

I have also had quite a lot of trouble with a number V1040 units and PLC lock ups.

I have used a contact of SB90 to trigger SB300 (PLC reset).

This is probably not recommended by Unitronics, but is the only thing at times to get me out of trouble.

I also use a closed contact of SB90 on all critical outputs to switch these off in the unit stops.

 

I like the fact that an error occurs if communication is lost to external modules the unit goes into "expansion error"

However if the PLC power is then cycled this error is reset and the PLC runs even with no external communication.

This is useful for troubleshooting but can be dangerous if the unit is not configured in a safe manner.

The ability for the PLC to run without the configured I/O should be an option that needs to be set for testing only.

 

Regards

 

Denis

[Joe Tauser]

+1 on what Denis says.  It would be really nice to have some SBs that report configured module status.

 

Joe T.

[Cara Bereck Levy]

Keith:

>

 

For the creators

Why does the internal watchdog built into the CPU of the V570 not ever figure out the CPU is locked and reset the CPU.  looking at the wiring diagram for the main processor it has the separate watchdog circuitry I would expect it to have.  Is this something that has been accidentally changed in a past firmware update and forgotten ever since? Is it an intentional choice....if so please explain?  I am not trying to shoot arrows, I love my Unitronics-products and they have worked well for many applications for years, but this is a serious condition that should be investigated. It is an easy fix in firmware (at least it appears to be) it just needs to be done. (or redone as it were)<

 

I'm sorry I missed this. Did you get help with this issue--did you contact Support directly? Please let me know.