Weird modbus TCP/IP behavior on one of my PLCs (V700)

[Fernando Castro]

Just to put all of you in context, I have a network of 180 Unitronics PLCs between v570 and v700.
eventually, every PLC will have the "same"  software (I am managing 2 versions of software one for each model)

There is a "server" (PC) that works as a Modbus master, and all the PLCs are Modbus slaves, the "server" continuously pulls data from the slaves
At the same time, there are other extra ports on the PLCs enabled for the Unitronics Suite (Remote operator, sd card explorer, download manager). All the PLCs IP are static and manually managed.
every Software compiled for the download has the same OS, so that has been standardized as well.

So far everything has been working fine but then ...

I have this issue with just 1 device that randomly stops responding to Modbus Master.

First I thought maybe another device was connected on the same network with DHCP enabled and was stealing that IP but I can connect using a remote operator.

so that rules out both IP settings and faulty cables.
I already tried to swap the physical "socket" connection just to rule out any configurations on the managed switch.
I already tried to use a different IP to test if by any chance the other machine happens to be a Modbus master using the same port and is trying to establish a connection to that specific IP (very unlikely In my opinion).

But in both cases still, the same PLC eventually stopped responding to my Modbus master. The funny thing is that I can restore the connection using the refresh button that I programmed on the PLC which closes the socket, restarts it, and run the Modbus config once again to restart the communication.

The only clue I have so far is that the error that the server shows is different from the one that I get from a disconnected device, disconnected devices eventually trigger timeout on requests. for this device, I get "unable to read".

Any Ideas?

I hate to say it because to me it always sounded like a lazy troubleshooting when I hear it but... I think maybe there is a problem with the PLC??

[Flex727]

After a thorough review of the software and other communications settings, as you have done, I would certainly try swapping out the PLC as the next step. Make a clone file on an SD card to install on the replacement PLC so that you know you have the exact same settings. If it fails again you know that it's probably not the hardware.

If you don't have a spare PLC (with 180 of them I would think you would have several), then just swap two that are in service to see if the problem moves with the software or hardware, once again using SD clone to swap the software.

Another option is to include the "refresh" routine in software to happen automatically.

[Fernando Castro]

47 minutes ago, Flex727 said:

Another option is to include the "refresh" routine in software to happen automatically.

That is a very good suggestion, I would assume that if there is no communication I should be able to see that on one of the network related SB 🤔 and try to reconnect whenever it happens.

 

[Flex727]

7 hours ago, Fernando Castro said:

I would assume that if there is no communication I should be able to see that on one of the network related SB 🤔 and try to reconnect whenever it happens.

Yes, just have a timer and if the Acknowledgements doesn't increment within a reasonable time then initiate your refresh routine.

[Ausman]

Fernando, all of Flex's stuff including swaparounds and last resort of make a self-repair fix.

I doubt this will help given what you have said, but have you checked the SB168 setting is correct?  I'd turn it off on that unit, do an init & reset, then do it all again with it back on.

I'd also be looking at 167 to see what it says when you have issues.

Perhaps that particular unit is near something like a VFD etc?  Checked that unit's earthing?  All cabling termination etc is good?

Have you checked in info mode that the port settings are definitely what you think they are, especially on a link failure?

cheers, Aus

 

[Gabriel Franco]

What I do is a kind of whatchdog: master writes continously 1 to an register (MI10); slaves resets it.

In the example, I use T50 to detect comms lost with master and act acordingly.

 

[kratmel]

Can you confirm or deny that the PLC that is having problems is the newest one in your system. I'm talking about the date of manufacture. I have one hypothesis about this.

Regarding trying to find the cause of the problem, the first step I would take is to replace the communication cable to the PLC. I have come across some PLCs not being able to switch from high speed to low speed in case of cable problems (I haven't found this yet with a Unitronics PLC though).

The second step is to replace the power supply in the problematic PLC. Check if the negative power terminal of this PLC is grounded. If not, then try to ground it.

The third step is to replace the PLC with a new one.

In such a large system, you should have at least 1 PLC of each type for hot swapping. Especially if all your PLC software is standardized.

[Fernando Castro]

Thanks all, I read a lot of interesting suggestions from you guys, regarding replacing the PLC and testing further more with that specific hardware, I would love to do it but  that will need to wait. since  its running production and everything else but the data logging over the network its working fine,  loosing that data until someone else hits refresh is a compromise that we are comfortable with, (for the moment).

There are other problems on production floor related to chain supply issues and other spare parts availability.

bottom line is:  If it ain't broke, don't fix it.

I can only say that the layout of the systems that we had …. is not great ...so I'll wait until a scheduled maintenance to replace that PLC.

The other day saw a maintenance guy replacing an entire cabinet to fix something that was clearly a loose connection, but it was just easier for them to replace the entire thing rather than opening the cabinet on site and troubleshoot. 😵

14 hours ago, kratmel said:

Can you confirm or deny that the PLC that is having problems is the newest one in your system. I'm talking about the date of manufacture. I have one hypothesis about this.

I sure can, as long is not invasive I can walk by production floor and check it. How Can I check this?, under info mode / version / hardware /  I guess?  or  if its available logging in with visilogic I don't even need to walk 🤣

 

20 hours ago, Gabriel Franco said:

What I do is a kind of whatchdog: master writes continously 1 to an register (MI10); slaves resets it.

In the example, I use T50 to detect comms lost with master and act acordingly.

This also solves other problem that I have been worried about. On my next software release I plan to implement Modbus Master on the PLC and ad other devices as Modbus slaves.

So far this was an isolated event, but I was worried that the same issue could break the connection for the other slave devices. 

[Flex727]

On 7/8/2022 at 8:32 AM, Fernando Castro said:

How Can I check this?

Serial No. is on the right-hand side (facing the screen), just above the Ethernet port.

[Ausman]

On 7/8/2022 at 11:32 PM, Fernando Castro said:

under info mode / version / hardware /  I guess

I don't think this is possible. Ive never really tried to link serials to system operands.   However, there are a lot of "un-named" elements in SDW, which might be a way of doing it.  For instance, SDW 9 is a unique number as shown, but I can't really relate that any way to a complete serial.  Maybe Unitronics has a cross-reference system, but whether they'd give it out is worth asking. But further down, 40 & 41 are different on various 130s I've just checked online, and they appear to never change, so maybe they relate?

@AlexUT  @Cara Bereck Levy    Is it possible to get serial no. via online looking?   Is there a facility to get a serial from SDW9?

cheers, Aus