Intermittent start-up problems

[Dave]

I have a machine with a Unistream 7" modular and some remote I/O attached to a URB-TCP. It has been working fine for  about 10 months., but now, occasionally when the machine is powered up, the PLC doesn't establish a connection with the URB-TCP.

The hardware model numbers are:

Panel: USP-070-B10

CPU:    USC-P-B10

I/O Adapter:   URB-TCP

When it does fail, all the status LED's on the URB are green, except the "Active" LED. I have checked the connections between the URB and the adjacent I/O module and the power supply connections on the front of the URB and they are all good. The cat5e cable between the PLC and the URB has also been verified as OK.

The boot-up of the PLC seems to take quite a long time... a little over 2 minutes.

Sometimes when it fails, not only does the remote I/O communication not get established, but also a couple of fields on my start up screen don't get populated with the related data. (I have the system tags "Panel SW Version.Unilogic Program Name" and "Panel SW Version.Unilogic Project Version" displayed on the start-up screen). It has, at times, booted up with these fields populated, but still no remote I/O communication.

I have not actually been there to hook up my laptop when it fails to see if there are any error codes. The customer "fixes" the problem by powering off/on until it boots up normally.

If anyone could shed some light on this behavior, it would be greatly appreciated. At this point, I'm thinking about replacing the CPU, but I am only guessing.

 

[Ausman]

To me this sounds like an intermittent connection issue, possibly related to ambient temp the machine has settled to whilst off.  I'd be going through every control and power supply connection in the thing, ensuring they are sound.  If the issue persists after that, then change power supply/ies and then finally the devices.  Don't do them all together as you'll never quite know what fixed the issue.

Is the environment dirty or humid or subject to large vibration?  Is the control gear well protected if it is?  It could even be bad connections inside the controller or I/O due to such things, and a wiggle here and there might help. 

Welcome to intermittent issue problem solving!

cheers, Aus

[kratmel]

 

2 minutes ago, Ausman said:

Welcome to intermittent issue problem solving!

Great invitation Aus!

By the way...

Is it possible that some VFD is installed near the Unistream installation?

I found some problem with communication and panel start if some  motor with VFD is operated in the same power net.

In my case GND wire of 24V power supply (-) connecting to machine GROUND solve problem.

[Dave]

Thanks for the feedback guys.

There are indeed VFD's in the area... not on the same equipment, but nearby. I guess I can try making sure they are off during power up and see if that helps.

I always ground the negative side of my power supplies, so if it is a VFD issue, install line reactors for the VFD's (other peoples equipment)? or line filter in my equipment, or both

The application is cosmetics, and the glitter does seem to get on mostly everything, but the controls are well protected in new, NEMA12/4 enclosures.

I have a second machine to do, so I have an order for the same hardware coming in. At least I'll have those to change out one by one, if it comes to that.

[Ausman]

The words glitter fill me with horror!  It could even be something on the I/O that has had ingress and is upsetting the power supply or connections.

And we all know how maintenance workers "never" leave enclosure doors open whilst running things, or after fixing something.  I once had a control enclosure that workers opened to isolate machinery whilst they knocked walls out and did lots of dirty, rearrange the site work.  Did they close the door after opening it?  Nup.  Did they then cover ALL the innards with the crap floating around that they were generating?  Yep.  Did it not work properly when they turned it back on?   And the prize goes to YES.  I've gradually realised over my lifetime that there truly are total idiots in the world that make me tear my hair out.

There have been countless stories on the forum where a client has a problem surface some time after the machinery is put into operation, and it's the OEM's fault.  After lots of over the phone work and trouble, the OEM eventually goes onsite and discovers new things added into enclosures, poorly done hookups, unclamped solenoids next to plc, obvious damage .....the list is endless.  If the VFDs are the problem, why is it just surfacing now?  More stuff put nearby perhaps?  Who knows.  Good luck.

cheers, Aus

[Dave]

Yes Ausman...many of my customers do indeed abuse their equipment, but in this case, they are very good at keeping enclosures closed and they regularly clean the equipment. While the glitter is everywhere on the machine itself, I found zero signs of debris in any of the electrical enclosures.

I had a CPU and panel brought in for another project, so I tried swapping out the CPU today...same problem occurred. When checking on-line status of the system tags, I found that  all numeric values were 0 and text values were "", empty quotes. (at least all the ones I looked at)

Then I swapped out the panel, using the original CPU, and the problem went away...so far. The client said that while it was intermittent at the beginning, as of last week, it was occurring almost every time they cycle power. They power off during every set-up change(as many as 2 or 3 times a day), as part of their safety procedure. I cycled the power three times today and had no issues.

I will give them a couple of days and if the issue has been resolved, I will send the panel back for warranty evaluation.

[Ausman]

Dave, wondering if you have checked the battery state?  Swapped it out for definite brand new only made last month blah blah?  

And a FYI laugh, if I go to a site with an intermittent issue, it almost always NEVER appears whilst I am there.  It is a standing joke that the systems don't play up because they know their Dad is there and they'll get into trouble.

Pls don't forget I'm not a Unistream user, only Vision, but many basic considerations remain the same.

cheers, Aus

[Joe Tauser]

On 7/7/2021 at 2:33 PM, Dave said:

Then I swapped out the panel, using the original CPU, and the problem went away...so far.

The panel is where the Ethernet control resides - a UniStream is a combination of two computers.  It sounds like the Ethernet controller went wanky - has a bad storm rolled through your user's location recently?  How far away is the URB-TCP?  I have a customer that recently lost a remote display due to surges on the communication line.

Joe T. 

[Dave]

 

Quote

The panel is where the Ethernet control resides - a UniStream is a combination of two computers

I was a bit surprised that when I first swapped the CPU out for a new one, the program was still intact when I powered up. I thought the program/configuration would reside in the CPU, but I guess it actually is stored in the panel. When I swapped the panel next, and used the original CPU, then I had to do a full download to restore the program. I wonder why the back-up battery is in the CPU and not the panel?

Oh, and the URB-TCP is only about 6 ft (maybe 8-10 ft of cable) away from the panel.

I did have a look at the battery status bits, but that was after a "mis-boot" and all the system bits were showing zeros. With the original CPU and new panel, the operating parameters for the equipment are being retained through power cycles, so I am guessing the battery is OK

I still have the suspect panel and I think I will power it up back at the shop with a new CPU /URB-TCP, and see what happens. It sure does help to have extra hardware around for this type of troubleshooting.

dave

[Dave]

Just a follow up on this....tried again with the suspect panel in the shop, with new CPU & URB. Sent it back to the distributor, who tried upgrading the firmware and got the same issues. The unit was returned to Unitronics and it was replaced under warranty, but no actual root cause was ever given.

[0x00]

was there any news here?  we stopped using the URB-TCP I/O and just installed a PLC with IO and connected with ModbusTCP.  The urbtcp was amazingly difficult to utilize and a whole productivity 2000, base, cpu and modules cost a bit less than all the same IO with the urbtcp......and you get processing ability to boot.  Also, the PLC lets you assign modbus addresses any way you like for the tags so you put all the ones you need in order and it's then easy to map to the HMI; no need to look up registers since you get to assign them all in a block if you want.

 

Was it really the panel?  I have struggled with the ethernet connections on these and also the 'adapter settings' in the windoze box but never been sure why the connections are so unreliable.

Our URB TCP never had comm issues, and I can connect 'through' the panel to other things downstream, but the programming connection always fails repeatedly.

[Ausman]

On 3/1/2022 at 6:55 AM, 0x00 said:

but the programming connection always fails repeatedly.

Hexfollower,   (!!)  as no-one else has chimed in yet, one thing that seems to crop up regularly, which may or may not be related, is the need to have the O/S and Unilogic versions exactly in sync for everything to work properly.

[greml1n]

I have similar issues with some Unistreams. I would get the “CPU lost communication” error message. Obviously the modules,would not boot when this occurred. Some power cycled reboots would eventually solve the issue but in some instances the customer would be rebooting 5-10x before the modules would boot. Tried upgrading firmware, didn’t help.  Then I systematically swapped each module with no success. Swapping the CPU did solve my issue in those circumstances though.

[Ausman]

12 hours ago, greml1n said:

Swapping the CPU did solve my issue in those circumstances though

So was this because the CPU was faulty, or because connections were being disturbed and made good in doing so?

[greml1n]

I assume the CPU was faulty as I checked CPU connections (not much to check). Recently had another facility getting the same issue, swapped CPU there and problem also went away

[Ausman]

I'd be carefully keeping track of failure times on this sort of thing.  There might be something fundamentally wrong in all the installs that the CPU  is not liking.  Like VFD interference, big spikes etc.

cheers, Aus