- A right alternative to IEEE-754's format - 6 Updates
- tuple (explicit?) constructors, overload resolution - 1 Update
| "Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Mar 29 05:46AM -0700 On Wednesday, March 28, 2018 at 2:00:59 PM UTC-4, gwowen wrote: > orders of magnitudes more precise than the measurements driving the > model -- doubly so when the process your modelling is the poster child > for sensitive dependence on those initial conditions. The initial value will be whatever the source data provides, but when you are dealing with calculations in series it is desirable to conduct your work out to a much greater decimal place, so that in each calculation (where you lose significant bits), that bit loss accumulation does not consume your desired significant bits at the level of your original source data. Your calculation shouldn't be a limiting factor in precision, even if your source data may be the limiting factor. 128-bits to 256-bits is able to handle nearly everything we need. But not in all cases, which is why I think it's important to provide an arbitrary precision numeric processing engine in a CPU, so the limitation on utility of use is not in the hardware. And, if done correctly, it will be faster than software libraries which do the same. And, as Terje Mathison points out in the comp.arch thread, there may be a way to provide features which enable a software version to drive the compute ability through a few new features added, which would then not require the numeric processor to do all the work, but just to be flexible enough to easily allow the software to use it for arbitrary precision calculations. > Back when I did a bit of it professionally, we'd run the same models and > multiple architectures, and the variation in FP behaviour was just > another way to add the ensembles. What does this mean? -- Rick C. Hodgin |
| David Brown <david.brown@hesbynett.no>: Mar 29 07:37PM +0200 On 28/03/18 17:27, Rick C. Hodgin wrote: > Weather modeling uses the double-double and quad-double libraries, > which leverage x87 hardware to chain operations to create 128-bit, > and 256-bit computations. That's fine - but weather modelling is not an example of "most uses". As I said, some cases need more about their numerics, but /most/ do not. > http://www.mpfr.org/pub.html > And John Gustafson discusses the shortcomings in his Stanford video > you didn't watch I posted above. I haven't bothered watching the video. I know that some people like them as a way of learning about a topic, but I usually find them extremely inefficient compared to a webpage or a paper that I can read at my own pace, spending more time on the parts I find difficult or particularly interesting. I have read a little of Gustafson's stuff on unums. I am not particularly impressed. As far as I can tell, it seems to consist of two ideas - making the number of significant bits variable, and allowing for a limited form of ranges. A variable number of bits makes hardware and software implementations painful and inefficient. Until you are getting to very large sizes, such as beyond octal precision IEEE (256 bits, or 32 bytes), it the simplicity of always dealing in a known fixed size outweighs the extra calculations needed for when a smaller bit length would be sufficient. The use of ranges sounds like a reasonable idea, but is only partly handled, and again it is more efficient to know that you always have a range pair rather than having to figure it out for each operation. There are definitely uses for other formats than standard floating point, or where you want to track ranges, error sizes, etc., as part of your number types. But any attempt at making a "universal number format" is guaranteed to fail - anyone who thinks they have made one, is wrong. IEEE floating point formats and calculation standards are a solid compromise between working for a wide range of uses and being efficient to implement, and have stood the test of time. They don't cover all applications, but no alternate format would do so either. |
| Gareth Owen <gwowen@gmail.com>: Mar 29 07:48PM +0100 > at the level of your original source data. > Your calculation shouldn't be a limiting factor in precision, even > if your source data may be the limiting factor. That's why I said "many orders of magnitudes". You need more precision, but you don't need about 64 (or more) bits of precision. >> multiple architectures, and the variation in FP behaviour was just >> another way to add the ensembles. > What does this mean? Well the first bit means "I used to work running weather modelling programs". So listen up. Due to the non-linear nature of fluid dynamics, the equations have sensitive dependence on initial conditions. So, imagine you start with (say) a sea-surface-temperature of 3.0 degrees at some point area your grid and run the code. Of course no-one measures temperatures to 3 decimal places - its idiotic - precision without accuracy. And even if you had perfect information, you're going to have to discretize that onto a fairly-coarse grid to model it, and now you've lost all the precision that no-one collects in the first place. Now imagine your discretisation/interpolation changes so you have an SST of 3.05 degrees in that area (which would be just a different interpolation of your temporally- and geographically-sparse observation). You run the code again, and in just a few days or weeks of time (in the model, i.e. predicting a few weeks ahead) the results have diverged *qualitatively* from your former run. This is exactly what is referred to as the butterfly effect, which was first described in weather modelling. But this sensitivity isn't just on initial conditions, its also on rounding. So you use the same inputs, and a different rounding mode -- or a processor that uses a weird 80-bit FPU -- and in a finite amount of time, your results are quantitatively different. So to get a good idea of what might happen you run the model multiple times with similar-but-different initial conditions, and on similar-but-different FPUs. This collection of models is called an ensemble, and they are "averaged" in some sense to give an overall estimate of what will happen, and bounds on how wrong that prediction is likely to be. |
| "Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Mar 29 12:13PM -0700 On Thursday, March 29, 2018 at 2:49:08 PM UTC-4, gwowen wrote: > > if your source data may be the limiting factor. > That's why I said "many orders of magnitudes". You need more precision, > but you don't need about 64 (or more) bits of precision. I think you need 1 bit per calculation, because every calculation will potentially introduce new rounding and decrease your precision. If you take a value through 64 separate calculation operations to arrive at the final value, you need the extra 64 bits to start with. Of course, as you say, it will greatly exceed the accuracy of your starting values, but that's a separate issue. Were you to improve your data set or modeling ability, then by having the extra precision already in there, nothing else would have to be changed. Each of the calculations would proceed as they did before, and now you'd have a better model. > ensemble, and they are "averaged" in some sense to give an overall > estimate of what will happen, and bounds on how wrong that prediction is > likely to be. Understood. I thought it was something like that with your use of the word "ensamble," but I didn't see how you were generating those "ensambles." I presume now there are the many models which then all get input and weight-averaged to produce a "central result" which the one broadcast on the evening news that night. :-) -- Rick C. Hodgin |
| Christian Gollwitzer <auriocus@gmx.de>: Mar 29 11:12PM +0200 Am 29.03.18 um 21:13 schrieb Rick C. Hodgin: > will potentially introduce new rounding and decrease your precision. > If you take a value through 64 separate calculation operations to > arrive at the final value, you need the extra 64 bits to start with. No, this is nonsense. Lookup "error propagation" or "uncertainty propagation". There is a simple textbook model to incorporate uncertainties of values into computations, by calculating a quadratic sum of the input errors times the partial derivatives. Simple examples for demonstration are catastrophic cancellation or square roots. Catastrophic cancellation means that you subtract two close values and get much less precision. E.g. if you know a=3.05 +/- 0.01, b= 3.00 +/- 0.01 then c = a - b = 0.05 +/- 0.014 SO even if ou knew a and b to 0.3% precision, a-b is known only to 20% precision. That's a loss of 6 bits of precision! OTOH if you take a square root, the number of significant figures increases by 1 bit. Yes - the square root is more precise than the radicand, which can be seen by expanding (1+epsilon)^(1/2) into a series. What Gareth describes is known as "Monte-Carlo error propagation" in metrology, and it is more accurate than the simple model, because it doesn't assume which statistical distribution the errors follow. Another way is to use the textbook model or special rounding rules to propagate the uncertainty with the data - this is known as interval arithmetics. Christian |
| "Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Mar 29 02:33PM -0700 On Thursday, March 29, 2018 at 5:13:06 PM UTC-4, Christian Gollwitzer wrote: > > If you take a value through 64 separate calculation operations to > > arrive at the final value, you need the extra 64 bits to start with. > No, this is nonsense. The results of every floating point calculation cannot be determined quite exactly. As a result, you take two values of 53-bit mantissa precision and perform some operation on them, and you then have a result where the 53rd bit is no longer certain. It has been rounded to the nearest bit, resulting in 0.5 bits of error max, but that last bit is no longer reliable. You're now down to 52. Repeat again, and that 53rd bit now affects the result which will be in the 52nd position, and both and 53 are now uncertain, leaving you 51 bits. And it continues. It's a side-effect of the last digit rounding on an FPU. The hardware designers have actually gone to great lengths to maintain greater bits internally so that when you perform a few calculations in succession, the rounded bits are beyond the ones that can be stored, therefore the final bits written out are all correct. Another such example is the fused_multiply_add operation introduced in IEEE-754-2008. It allows a multiply and an add to take place without intermediate rounding. This results in a value using FMA which will nearly always be different than if multiply, then add operations were performed in serial. Floating point is a good approximation of real numbers, but it has many flaws. If you want to maintain precision over many successive calculations in series, you need to have high precision computation so the ULPs being affected at each stage are way out there, and not close to your target bit requirement for your required precision when converted to base-10. > doesn't assume which statistical distribution the errors follow. Another > way is to use the textbook model or special rounding rules to propagate > the uncertainty with the data - this is known as interval arithmetics. I'm only speaking about FPU hardware losses of precision. There is the last bit in computations called the ULP. One half the ULP is potentially lost with each calculation depending on whether it can be represented accurately or not. https://en.wikipedia.org/wiki/Unit_in_the_last_place https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html Once you lose the guarantee of reliability of the last bit, the next calculation cannot be accurate to the bit before that, and so on. In many cases, however, the rounding will round up here, and down there, resulting in a reasonable answer out to most mantissa bits. But you can construct purposeful series that will demonstrate the loss significantly. -- Rick C. Hodgin |
| Latimerius <l4t1m3r1us@gmail.com>: Mar 29 09:18AM -0700 Hello, please consider the following code: #include <functional> #include <tuple> #include <iostream> struct C { using Fn = std::function <void (int )>; Fn mFn; C (Fn && fn) : mFn (std::move (fn)) {} C (std::tuple <Fn> && fn) : mFn (std::move (std::get <0> (fn))) {} }; int main () { C c1 ( [] (int i) { std::cout << i << std::endl; } ); } It's not compilable as the constructor call in main() is apparently ambiguous. Two things that I don't understand about this: 1) why does the second C's ctor even apply? Shouldn't the corresponding std::tuple ctor be explicit? 2) assuming both C's ctors are applicable - shouldn't the first one be a better match? I guess it only requires a single conversion (from lambda to C:::Fn) whereas the second one requires two (lambda to C::Fn as before and then from C::Fn to std::tuple). Thanks in advance! |
| You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment