decoration decoration
Stories

GROKLAW
When you want to know more...
decoration
For layout only
Home
Archives
Site Map
Search
About Groklaw
Awards
Legal Research
Timelines
ApplevSamsung
ApplevSamsung p.2
ArchiveExplorer
Autozone
Bilski
Cases
Cast: Lawyers
Comes v. MS
Contracts/Documents
Courts
DRM
Gordon v MS
GPL
Grokdoc
HTML How To
IPI v RH
IV v. Google
Legal Docs
Lodsys
MS Litigations
MSvB&N
News Picks
Novell v. MS
Novell-MS Deal
ODF/OOXML
OOXML Appeals
OraclevGoogle
Patents
ProjectMonterey
Psystar
Quote Database
Red Hat v SCO
Salus Book
SCEA v Hotz
SCO Appeals
SCO Bankruptcy
SCO Financials
SCO Overview
SCO v IBM
SCO v Novell
SCO:Soup2Nuts
SCOsource
Sean Daly
Software Patents
Switch to Linux
Transcripts
Unix Books

Gear

Groklaw Gear

Click here to send an email to the editor of this weblog.


You won't find me on Facebook


Donate

Donate Paypal


No Legal Advice

The information on Groklaw is not intended to constitute legal advice. While Mark is a lawyer and he has asked other lawyers and law students to contribute articles, all of these articles are offered to help educate, not to provide specific legal advice. They are not your lawyers.

Here's Groklaw's comments policy.


What's New

STORIES
No new stories

COMMENTS last 48 hrs
No new comments


Sponsors

Hosting:
hosted by ibiblio

On servers donated to ibiblio by AMD.

Webmaster
About binary representations | 661 comments | Create New Account
Comments belong to whoever posts them. Please notify us of inappropriate comments.
About binary representations
Authored by: coats on Friday, March 29 2013 @ 12:57 PM EDT
You had better make that "convert to integer" instead be "round to integer."

And "converted to binary" may itself be problematic: some systems (Intel x87, IBM POWER) have different binary floating point formats "on the processor" vs "in memory" -- for the x87, on-processor arithemtic is 80-bit and in-memory is either 32-bit or 64-bit The instruction for "store to memory" is really "round 80-bit floating point to (less precise) memory format and store the result". For IBM POWER version 4 or earlier, on-processor is 64-bit while in-memory is either 32-bit or 64-bit. For DEC VAX, there were two distinct 64-bit formats, one with a large exponent-range and about 17 significant digits of fraction, the other with a smaller exponent-range but about 20 significant digits of fraction.)

Consequently, especially with optimized code you can get surprises like the following:

  1. X = 5.0/3.0
    and store to memory as 32-bit
  2. ...
  3. Y = 5.0/3.0
    never stores to memory
  4. IF ( X equals Y ) THEN...
    Fetch 32-bit value for X from memory, compare with 80-bit on-processor value for "Y", and find that the two are unequal
  5. ...code that never uses Y again...
Furthermore, if you compile with full debug flags, results are always stored to memory, and you will get the result that naive users expect... but much slower, which makes a difference if you write stuff like I do (e.g., weather models) that runs for hours at a time on supercomputers ;-(

[ Reply to This | Parent | # ]

Groklaw © Copyright 2003-2013 Pamela Jones.
All trademarks and copyrights on this page are owned by their respective owners.
Comments are owned by the individual posters.

PJ's articles are licensed under a Creative Commons License. ( Details )