Jump to content

Recommended Posts

  • MVP 2021

I rarely use floating point operands, but my understanding is that it is stored in a 32-bit memory location, just like an ML, but it has to store both the significand and the exponent in that 32-bit location. The situation with MF is that it trades off between range and precision. You cannot get 10 digits of precision in an MF.

Link to comment
Share on other sites

  • MVP 2021
4 hours ago, Flex727 said:

The situation with MF is that it trades off between range and precision.

Unitronics uses single precision floating point-

https://en.wikipedia.org/wiki/Single-precision_floating-point_format

As described in the above link, once you get above 9 digits (as you show in your example) the magical exponent shift lever kicks in and knocks the number of significant digits down to 7 to allow for the exponent value.

Which is why cleverly manipulated long integers are more accurate and precise than floating point in many applications.

Joe T.

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...