Search the Community
Showing results for tags 'accuracy'.
Found 1 result
I'm trying to accurately calculate the cosine of an integer to at least 4 decimal places. For example if I use 12000: When I use Excel with the formula =COS(12000) it returns 0.634112 When I key the values into my desktop calculator it returns 0.63411204 (must set Radian mode). In VisiLogic 9.3, I used the FP function COS and got 0.6338002. Using the Formula block with "1000 * (COS(12000))" (stored to an MI) returns 633. Can anyone share some insights? Has anyone developed a work-around?