Facing precision issues with "double"

microsoft.public.dotnet.general

7 Jun 2005 9:58 AM

Uma Abhyankar

Hello,

We are facing precision issues with addition, multiplication or division of
"double"
Issue1:
#####
The output of 0.1 + 0.2 is not 0.3
It is 0.30000000000000004
How do we tackle this??

Issue2:
#####
Minute value: 8.6994833333333332
Case1: From Minutes to Seconds:
=========================
Operation: 8.6994833333333332 * 60
Expected output = 521.968999999999992
Actual output = 521.969
-Give output of Case 1 to conversion again

Case2: From Seconds to Minutes:
=========================
Operation: 521.969 / 60
Expected output = same as input i.e. 8.6994833333333332
Actual output = 8.6994833333333333333333333333333
[As the input to Case2 was 521.969 and not 521.968999999999992]
How do we tackle this?

Thanks & Regards
Uma

7 Jun 2005 10:03 AM

Cor Ligthert

Uma,

I have seen some more questions about this in this newsgroup.

I have searched for you un this newsgroup, where I took only one replyer.

http://groups-beta.google.com/group/microsoft.public.dotnet.general/search?group=microsoft.public.dotnet.general&q=skeet+double&qt_g=1&searchnow=Search+this+group

I hope this helps,

Cor

Are all your drivers up to date? click for free checkup

7 Jun 2005 4:31 PM

Jon Skeet [C# MVP]

Uma Abhyankar <a***@persistent.co.in> wrote:

> We are facing precision issues with addition, multiplication or division of
> "double"

See http://www.pobox.com/~skeet/csharp/floatingpoint.html

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too