Jump to content

Recommended Posts

I'm using V280 with Visilogic 9.8.64 build0

I have the simple formula A(ML)x1=B(MF), I try A=2147483647 but the output = 2.147484E+9 why it's not 2.147483647E+9 because in the manual said maximum MF range is 3.402E37

Please advise, thank you.

Share this post


Link to post
Share on other sites

I rarely use floating point operands, but my understanding is that it is stored in a 32-bit memory location, just like an ML, but it has to store both the significand and the exponent in that 32-bit location. The situation with MF is that it trades off between range and precision. You cannot get 10 digits of precision in an MF.

Share this post


Link to post
Share on other sites
4 hours ago, Flex727 said:

The situation with MF is that it trades off between range and precision.

Unitronics uses single precision floating point-

https://en.wikipedia.org/wiki/Single-precision_floating-point_format

As described in the above link, once you get above 9 digits (as you show in your example) the magical exponent shift lever kicks in and knocks the number of significant digits down to 7 to allow for the exponent value.

Which is why cleverly manipulated long integers are more accurate and precise than floating point in many applications.

Joe T.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×