|
Hi,
(1) I have met several cases where epsilon is added to a non-negative variable to guarantee nonzero value. So I wonder why not add the minimum value that the data type can represent instead of epsilon? What are the difference problems that these two...
Started by Tim on
, 4 posts
by 4 people.
Answer Snippets (Read the full thread at stackoverflow):
In the case of double precision, the difference between the maximum and the next, there is no strict answer, it depends....
It depends on the precision of the floating point number of course.
Is actually 1e-16 while the second is approx 1e+11.
|
|
Hi All,
Just wondering what the difference between a signle precision floating point operation and double precision floating operation is.
I'm especially interested in practical terms in relation to video game consoles, for example does the nintendo 6...
Started by meds on
, 7 posts
by 7 people.
Answer Snippets (Read the full thread at stackoverflow):
The number of bits in double precision increases the maximum value that can be stored as well....
Basically single precision floating point arithmetic deals with 32 bit floating point numbers whereas double precision deals with 64 bit.
|
|
I'm using long double in a C program to compute 2D images of the Mandelbrot Set but wish to have further precision to zoom deeper.
Are there any performance gains to be had from an arbitrary precision maths library that can restrict the amount of precision...
Started by James Morris on
, 3 posts
by 3 people.
Answer Snippets (Read the full thread at stackoverflow):
If you need more precision, see qd at http.
This wiki article contains links to several libraries.
|
Ask your Facebook Friends
|
I want to get 1/7 with better precision, but it got truncated. How can I get better precision when I convert a rational number? Thanks.
>>> str(1.0/7)[:50] '0.142857142857'
Started by grokus on
, 5 posts
by 5 people.
Answer Snippets (Read the full thread at stackoverflow):
Python's floats are never arbitrary precision....
EDIT) 14285714285714285714285714285714285714285714285714L
Python's integers are arbitrary precision.
You could multiply the numerator by a large 10^N and stick with arbitrary-precision integers.
|
|
Is there a way to set Perl script's floating point precision (to 3 digits), without having to change it specifically for every variable?
Something similar to TCL's:
global tcl_precision set tcl_precision 3
Started by Igor Oks on
, 3 posts
by 3 people.
Answer Snippets (Read the full thread at stackoverflow):
BigFloat or bignum ?
use Math::BigFloat; Math::BigFloat->precision(-3); my $x = Math::BigFloat->new.
|
|
Is there a way to alter the precision of an existing decimal column in Sql Server?
Started by Andy J on
, 4 posts
by 4 people.
Answer Snippets (Read the full thread at stackoverflow):
I haven't done any testing ....
You will have to modify your decimal(precision, scale) replacing the precision and scale with your desired values.
In the properties at the bottom there is a precision property
None that I'm aware of.
|
|
Hello, can you tell me how can I get a high precision sleep-function in Python2.6 on Win32 and on Linux?
Started by Richard J. Terrell on
, 3 posts
by 3 people.
Answer Snippets (Read the full thread at stackoverflow):
However, it's unlikely that you will get much more than millisecond precision with sleep because operating.
|
|
Why is the precision of my location in Mapkit not as good as in the maps app? The accuracy is set to best. If i'm trying to get my location in maps.app i have an acc of 50m in my app i habe 500m. What am i doing wron :)? Any tipps to get a better result...
Started by rdesign on
, 4 posts
by 4 people.
Answer Snippets (Read the full thread at stackoverflow):
GPS is very sensitive to obstructions above you, and might even be disturbed by wires etc that are connected... .
The only thing I can think of, under the assumption that you have looked through all references, is that you test them under different conditions .
|
|
I'm interested in implementing an algorithm on the GPU using HLSL, but one of my main concerns is that I would like a variable level of precision. Are there techniques out there to emulate 64bit precision and higher that could be implemented on the GPU...
Started by Mark on
, 5 posts
by 5 people.
Answer Snippets (Read the full thread at stackoverflow):
There are a wide variety of techniques that have composed of multiple floats in whatever... .
GPUs are just beginning to support double precision in hardware, though it will continue to be much slower than single precision in the near future.
|
|
Precision loss is one thing, but precision gain???
I have a text file w/ the following coordinates:
41.88694340165634 -87.60841369628906
When I paste this into SQL Server Mgmt Studio table view, it results in this:
41.886943401656339 -87.6084136962890...
Started by Scott on
, 4 posts
by 4 people.
Answer Snippets (Read the full thread at stackoverflow):
It's not adding precision to decimal, it only LOOKS like it gained precision..
Remember that floats are all approximate anyway.
Be stored in the db.
|
