Friday, November 22, 2019

Digest for comp.lang.c++@googlegroups.com - 14 updates in 6 topics

Vir Campestris <vir.campestris@invalid.invalid>: Nov 22 09:46PM

On 19/11/2019 11:48, Alf P. Steinbach wrote:
 
> But I would guess that any group of C++ programmers using deduced return
> type is not yet large enough that there would be any herd followers.
 
> - Alf
 
That's interesting. That sound as if you're on my side of the "almost
always auto" argument I've heard bandied about - not least by the guy
who owns our coding standards :(
 
The argument is that given "x=4.2" you _know_ that x is a double,
there's no point in saying so.
 
I don't like it - that variable ought to be a type you know, and you
should say so. I find it easier to _read_ code that explicitly states
the types. OTOH, I've experimented, and it's easier to _write_ code that
uses auto everywhere.
 
When I read "auto x = somefunction()" I don't know what x is. Unless I
look at the declaration. Which if I'm unlucky is one of several
overloads with template type deduction from the parameters.
 
But IMHO reading matters more. You'll only write it once.
 
Andy
David Brown <david.brown@hesbynett.no>: Nov 22 04:34PM +0100

On 22/11/2019 16:23, Öö Tiib wrote:
 
> There we slightly mixed up in our thinkings ... I myself imagined
> something like array of "quantities" where double is value and char
> selects its magnitude (or physical unit) from limited set of such.
 
OK. The gains from having these as separate arrays (rather than packed
structs) might not be as much, but they could still be relevant.
 
> storage optimization. But it can also increase cache misses
> because locality of members of object is now gone ... needs to
> be profiled.
 
You still have as much locality, especially when iterating over the
arrays, just split into two blocks. Random access might be worse - you
will always have to pull in two cache lines, rather than only sometimes
needing to do so. (But if you are doing mainly random access and speed
is important, unpacked structs will be best.) Misalignment penalties
for the packed struct can vary from nothing to a great deal, depending
on the target platform.
 
Yes - you need to profile and test.
 
James Kuyper <jameskuyper@alumni.caltech.edu>: Nov 22 10:49AM -0500

On 11/22/19 4:10 AM, David Brown wrote:
> On 22/11/2019 09:37, Keith Thompson wrote:
...
> end (beyond what is needed for an array of the struct to have proper
> alignment for all members), are there any compilers that have done so in
> practice?
 
I can't name names, but my understanding is that there have been (and
might still be) implementations where all struct types have the same
alignment requirement as the most strictly aligned type supported by
that implementation. If that alignment requirement is greater than the
requirements of any of the member types, that will sometimes require
padding at the end of the struct.
James Kuyper <jameskuyper@alumni.caltech.edu>: Nov 22 10:52AM -0500

On 11/22/19 10:49 AM, James Kuyper wrote:
> that implementation. If that alignment requirement is greater than the
> requirements of any of the member types, that will sometimes require
> padding at the end of the struct.
I said that wrong. Unless all members of the struct have the maximum
alignment requirement, it's possible that such an implementation would
have to put padding at the end.
James Kuyper <jameskuyper@alumni.caltech.edu>: Nov 22 10:56AM -0500

On 11/22/19 4:37 AM, Öö Tiib wrote:
> On Friday, 22 November 2019 10:37:59 UTC+2, David Brown wrote:
...
>> full size, packed means you can get misalignments, which can have a very
>> significant impact on efficiency in some situations.
 
> Huh? Packed structs usually lead to raise of storage efficiency.
 
He said efficiency, not "storage efficiency". If there's any difference
between the packed and unpacked version of a struct, the packed version
usually requires slower access. If that weren't the case, there'd be no
motive for the implementation to make the unpacked version different.
Manfred <noname@add.invalid>: Nov 22 05:13PM +0100

On 11/22/2019 12:38 PM, David Brown wrote:
> appropriate accessor functions. On the plus side, that would let you
> make an endian-independent solution, and if it is done well the
> compiler's optimiser will eliminate all overhead.
 
It is defined behavior in C++ too, but you have to pay extra care
(compared to C) in order to ensure that only the active member is accessed.
 
David Brown <david.brown@hesbynett.no>: Nov 22 06:48PM +0100

On 22/11/2019 17:13, Manfred wrote:
>> compiler's optimiser will eliminate all overhead.
 
> It is defined behavior in C++ too, but you have to pay extra care
> (compared to C) in order to ensure that only the active member is accessed.
 
I was assuming Öö meant that type punning via the union - accessing the
same data via the different union members - was undefined behaviour (as
it is, in C++). Using the union while only accessing the active member
is of course well defined - otherwise the union concept would be pretty
useless!
 
But it would have been worth making that point more clearly.
 
Keith Thompson <kst-u@mib.org>: Nov 22 10:00AM -0800

> "dot" member selector not only clearer (more readable) but
> more amenable to "intellisense" when typing (type less, no
> typos as autocompletion pops up the members).
 
If you have a struct consisting of 4 uint8_t members, it's very
likely that you don't *need* #pragma pack. There might be some
odd compiler out there that allocates more than 32 bits for such
a struct -- but that odd compiler might not support #pragma pack.
 
struct pixel {
uint8_t red;
uint8_t green;
uint8_t blue;
uint8_t alpha;
};
 
If you need to write 100% portable code, that might not be good
enough -- but you very likely don't need to write 100% portable code.
(And if you're willing to consider #pragma pack, it seems you
already know that.)
 
My point in suggesting an array was simply to approach that 100%
portability, but I don't think it's necessary.
 
[...]
 
--
Keith Thompson (The_Other_Keith) kst-u@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */
Soviet_Mario <SovietMario@CCCP.MIR>: Nov 22 07:35PM +0100

Il 22/11/19 15:04, David Brown ha scritto:
 
>> sure I have no idea, I put the pragma there just for "precaution"
>> what cost is that supposed to have ?
 
> I've told you in several posts what the costs are.
 
put is alas irregular verb : I meant its PAST form actually.
I have abandoned the unsigned Bit_Field : 8 solution as soon
as you (a lot of people) stated that u_int_8 MUST have
exactly 8 bits.
 
In that case either pragma is USELESS (and hence does not
produce any reasonable effect, neither slowness), either is
NECESSARY (and so the slowing effect is a unavoidable price
that I MUST pay either)
 
> "Packed" is /not/ a
> "precaution", nor is using a struct with a bitfield instead of uint8_t.
 
the second part has been abandoned early
 
> If you don't know what you are doing
 
I'm rather weak in higher level C++ constructs, and I know
that, but not particularly so in low level (some couple of
tens of years ago I used to stuff C code with inline "asm"
statements or even __emit() calls).
 
>, fine - if I understand you
> correctly, you are experienced with gambas
 
no, I am rather new to gambas actually.
I was used to VB (Visual Studio, but not even recently).
Going to Gambas has been actually a pain (not saying it is
bad, absolutely not : it's me that knows much less than .net
framework). On QT I am even more newby.
 
> programming but new to C++.
 
well, I am more of outdated than new, I had used C++ (and C)
a lot in 1995-2000 roughly, then less and less afterwards.
But strangely enough, I find it more easy to think in C++
terms when I met problems in basic.
 
> So /ask/ (and please read the answers). Don't guess, or grab the first
> thing that you find from a google search.
 
but I DID follow the answers :)
 
> And when the more experienced
> people in a group like this all say "don't use packed" and "use
> uint8_t",
 
I didn't read or desume the two things should be mutually
exclusive.
 
Anyway, there are three scenarios
 
1) added pragma (to the four u_int_8 members in a struct) is
USELESS, as the struct was just packed natively (and hence
does not produce any reasonable effect, neither slowness),
either
2) is NECESSARY, as the struct would not have been natively
packed otherwise (and so the slowing effect is a unavoidable
price that I MUST pay either)
either
3) it is NOT SUFFICIENT (in this case, luckily unlikely,
nothing would have worked fine at all)
 
as for ALIGNMENT problem, I should not meet any I guess.
The packed (natively or forcefully) struct is 32 bit wide,
so allocating an array should allow nice addressing of any
item even on a 64 bit OS
And no item would cross critical addressing boundary
 
 
> then that is what you should do. Ask /why/, and try to learn
> - and once you do understand "pack", when it can be useful, when it is
> useless, when it is worse than useless,
 
If you have some further patience, tell me if the 3
scenarios fit the present situation (not a more general one,
but just this)
 
> what it does, and what it costs
> - /then/ you can decide that because you know the full situation better
> than we do, you still want to use it against general advice.
 
I had given all details sufficient : the data layout is what
I said, so no obscure aspects
 
 
> "Packed" is not a precaution -
 
I call it a precaution as it reduces the chances of a non
packed data layout. I was not saying it would be successful
necessarily in every scenario (even sizes sequence is
relevant, and I know this, but my members were all 1 byte long)
 
> have nothing better and know you can't write the code as well without
> it. (That might mean you can't write the code as quickly or clearly
> without it, which is fair enough - /if/ you are sure it is necessary.)
 
let's say it explicitely
 
if I have
union
{
struct
{
u_int_8 Red, Green, Blue, Alpha;
};
u_int_32 AllChannels;
};
 
without PRAGMA, do I have any explicit or implicit warranty
that my union be 32 bit long ? If so, PRAGMA is useless. If
not, it seems it might be useful.
what damages could PRAGMA PACK produce ?
If the union would be just packed natively, I guess none.
If it wasn't, at best it would pack it (and maybe slow it,
but this is irrelevant, I need precise disk layout). At
worsk it would be insufficient to actually pack it, so my
problem is unsolved.
In this last worst case, are there any other means of
getting the thing to work ?
What are them ?
 
 
>> operations
 
> Of course there is use in knowing this, if speed is an issue. If the
> processing takes 1000 times as long as the loading and saving,
 
this could be true only if load/store is "optimized" like I
stated. If it was done an item at a time it would become
very very slow (and depending on smartness of OS caching
anyway, but not even so much even then, as I read/write the
whole file, which can be 4-12 MB long and one time only, so
caching cannot be as useful as when reusing a small region
many times).
 
> then you
> know that putting the slightest effort into speed optimisation for file
> access is a waste of effort
 
when I made some elaboration all in gambas the times were
"mixed" : some basic transformations with few or no floating
point calculations went actually FASTER than disk
read/write, with some more f.p. intensive calculations disk
load/save were faster.
But in any case, the same "order of magnitude", not
100-1000-etc times slower or faster.
 
> shave 0.1% off the processing time than increasing the speed of the file
> access by orders of magnitude.
 
> And if your file format contains non-aligned members,
 
why making general guess when I reported precisely the layout ?
I said :
16 bit (width) - 16 bit (height) - N x 32 bit (R,G,B,alpha)
where N = width x height = number of "pixels"
 
every pixel is multiple of 32 address on file (but this
seems irrelevant to me).
In the ram layout,
the BMP descriptor has width and height as "local" members,
and a pointer to an array of struct aligned in memory by the
NEW [] operator who is aware of alignment requirements.
 
Where are the possible problems ?
What problems are affected by a pragma pack directive ?
 
 
> then the simplest
> and easiest way to improve and simplify the code, and improve the speed
> of several parts, is to fix the file format.
 
that point is unmodificable. I strictly need full color and
no compression. The alpha channel is maybe a bit overkill
(255 level of transparency is more than needed ... but
taking less is useless or impossible, and 1 byte is natural
for the otherwise 3,sth bytes long struct)
 
> Make sure the members are
> aligned (add explicit dummy members if necessary)
 
since when reading/writing files suffer from this ? Files
are BYTE oriented internally (and the OS does caching
independently and transparently, basing on disk cluster
sizes but this is not sth I should care of)
 
as for memory : I instance the BMP array invoking
ptrBMP = new clsMyStructColor [TotSize];
and I expect the structs are aligned correctly as NEW knows
its task.
 
>> disk layout are the very same raw binary.
>> I was asking help in making sure this was (not if it was useful or not)
 
> No one can tell you if it is the same without knowing the file format.
 
 
I just said it : the stupidest format of all
 
16 bit (width) - 16 bit (height) - N x 32 bit (R,G,B,alpha)
where N = width x height = number of "pixels"
 
 
 
> There is nothing wrong with having the ram and disk layout the same -
 
oh happy to know ! :)
 
> layout is not directly usable in ram, then it often is /not/ a
> convenient and efficient choice - and if it /is/ usable directly in ram,
> you don't need or want "pack".
 
I was asking about the specific case
 
 
> However you look at it, "pack" is usually (but not always) the wrong
> choice. You should only consider it once you have eliminated better
> alternatives.
 
for a starting, I tried to use a 32 bit total size, hoping
it would have been packed regardless of any "hints" (pragma).
 
 
> As for asking for help here, you are getting the help you need even if
> it is not exactly what you asked for.
 
not a problem, every advice is appreciated
 
>> of different layout, in a single read operation ? I can't figure out this
 
> First, you take a step back. You look to see if the packed file really
> is "packed",
 
it is, as both Gambas supports BYTE format (And sizeof())
and BYTE-oriented streams. No overspace is generated. Then I
have to worry about the C++ back end, to comply with this fact.
 
I think the designer of Gambas relies very strongly upon C
libraries and mind-shape, apart from strings (where he had
to welcome BASIC users who want strings the basic way).
 
> in the sense of having unaligned data that would have
> padding in a normal struct. I have seen no answer to this (unless it is
> later in this post), despite asking this /crucial/ question.
 
the files are packed.
 
 
> If the format is already good, you are done. One memcpy, or read from
> the file directly to a suitable buffer, or memmap to the file directly
> (no reading or writing needed).
 
memmap is a problem in that I don't know how exactly works
and whether or not I'll be able to interoperate with memory
files opened in gambas
 
I am also considering to use some memory files supported
directly by the OS (here on linux files created on /shm
virtual folder are designed for such FAST temporaries to be
shared, but with portable stream syntax).
Using /shm would imply no specialized load/store code, just
the normal file access => delegating the ram usage to the OS.
 
 
> If the format is bad, the next step is to consider changing the format.
 
just observation.
So, the format is wrong ?
 
 
> If the format is bad, and cannot be fixed, then you have to write reader
> and writer functions that unpack and pack the data into the right
> formats. Reader and writer functions are common for all sorts of
 
well, I'd be considering these for small size, complex
structured input, not for a 4-12 MB size of simple pixels.
It would be a blood bathe
 
> header and then allocating memory depending on the size. It is very
> normal to have a function that reads in the data format and uses it to
> populate an internal format.
 
in fact, minimally, I do this. First create the container,
with just a couple of members, then depending on their
value, size the buffer and allocate it, then load the file
in it.
 
> Perhaps your file format uses 8-bit
> unsigned data for colour channels, but your internal format uses
> floating point because that suits your processing needs.
 
On certain routines I use floating point, but only
temporarily, as I never STORE floating point pixels. I don't
need a huge precision, as operations are not supposed to be
applied multiple times on the same picture, creating
rounding additive errors.
 
> expand it to a raw array. There are all sorts of things that go on in
> typical reader functions. Making them a "single operation", whatever
> that might mean,
 
sth encoded in a single call to fread / fwrite, at least for
the pixel data block. The header is distinct
 
> is rarely relevant - code that uses these functions has
> a "single operation" of calling the reader function.
 
well, fread/fwrite may maybe internally operate differently
than expected, but in this case I would not be able to do
better I guess :)
 
 
> of course.
 
> (And to be clear - I did not right this to be rude, or insulting, or
> demoralising, and I hope it did not come across that way.
 
no problem, really
 
 
>> I don't know. Just searching a way to do binary read/write in one single
>> shot directly to/from RAM without any conversions.
 
> Doing this all "in one single shot" should not be an aim. Doing it in a
 
I wish to both get fast disk I/O but also limit the
try/catch trapping. I have the habit to catch or check errno
on every disk access. A single huge read/write makes the
code thinner than splitting the accesses in many place and
having to multiplicate the error condition testing.
 
>> the pillar.
 
> Fair enough. So get it /right/ by designing the right data structure
> /now/, rather than picking a bad structure from the start.
 
please give your judgement about the struct I have described
before, which drawbacks seems to have or don't have.
 
> Start with a member of that size for convenience - I like a constant
> marker to make it easy to tell that you are using the right file type.
> (The value here is the string "Picture".)
 
I have stripped everything non necessary as this raw format
is only a temporary. When I actually store files, I export
them as PNG of JPG or GIF by Gambas code.
So my internal format was really minimal.
 
>> personal) format for intermediate raw data.
>> It does not seem to be more sophisticated than this.
 
> Then it is inexcusable to have a format that needs "packed". That just
 
oh I completely disagree with this point ! The format has to
be packed as the C++ library receives the file from an other
source of data that offers them in that form, so it's mandatory.
And also the caller expects to get the transformed data back
in the same packed form.
 
> means totally pointless complications, inefficiencies, and non-portable
> code.
 
nope, I have constraints to fullfil. Maybe inefficient,
maybe non portable (who cares ?), but surely not pointless,
as otherwise data exchange would not work at all
 
 
--
1) Resistere, resistere, resistere.
2) Se tutti pagano le tasse, le tasse le pagano tutti
Soviet_Mario - (aka Gatto_Vizzato)
James Kuyper <jameskuyper@alumni.caltech.edu>: Nov 22 02:11PM -0500

On 11/22/19 1:35 PM, Soviet_Mario wrote:
> Il 22/11/19 15:04, David Brown ha scritto:
...
> I have abandoned the unsigned Bit_Field : 8 solution as soon
> as you (a lot of people) stated that u_int_8 MUST have
 
uint8_t
 
> exactly 8 bits.
 
It must have exactly 8 bits, if it exists. uint8_t is optional. And on
any C99+ implementation that fails to support uint8_t, and on any C
platform where unsigned char doesn't have 8 bits, you can't do what you
want to do anyway.
 
> produce any reasonable effect, neither slowness), either is
> NECESSARY (and so the slowing effect is a unavoidable price
> that I MUST pay either)
 
Correct - it is both useless and unnecessary - in this context. Don't
extrapolate from this case to the general case. It is very specifically
the fact that you are trying to pack something that cannot possibly be
bigger than a byte, and therefore cannot have an alignment requirement
bigger than 1, that renders the pragma useless and unnecessary.
 
> that, but not particularly so in low level (some couple of
> tens of years ago I used to stuff C code with inline "asm"
> statements or even __emit() calls).
 
You don't appear to understand why the fact that you're trying to use an
8-bit type guarantees that #pragma pack will be useless and unnecessary
- so there's some low-level stuff you don't understand yet.
 
> either
> 3) it is NOT SUFFICIENT (in this case, luckily unlikely,
> nothing would have worked fine at all)
 
On implementations where CHAR_BIT==8, scenario #1 applies. On
implementations where CHAR_BIT>8, scenarios #1 and #3 both apply.
 
...
> u_int_32 AllChannels;
 
uint_32
 
 
>> padding in a normal struct. I have seen no answer to this (unless it is
>> later in this post), despite asking this /crucial/ question.
 
> the files are packed.
 
OK - which fields are positioned differently from the way they would be
positioned if not packed?
Paul <pepstein5@gmail.com>: Nov 22 09:31AM -0800

I solved the following problem:
Given a sequence in the form of a vector<int> and a non-negative integer r,
find the length of the longest subsequence which has range <= r.
By "range", I mean the stats definition of max - min.
 
The code is below.
I solved it by hashing as this is much faster than sorting for long vectors.
(I tested for vectors of size >= 10 million).
I am particularly interested in small r, and the most interesting case to
me is r == 1 because that case was given to me as part of a job interview
process many years ago.
 
I was advised that the relevant hashtables should not be allowed to grow
by introducing a new element when the key is not present.
However, when I decommented the below line to prevent the hashtable
growing, I saw no change in performance.
Should the commented code be commented or decommented?
Any other feedback is welcome, too.
 
Many thanks.
 
Paul Epstein
 
/* CODE BELOW ******************************************************/
 
int hashLongest(const std::vector<int>& vec, int r)
{
// For each int in the vector, count how many times it occurs.
std::unordered_map<int, int> allCounts;
 
for (int v : vec)
++allCounts[v];

// For each int k in the vector, count how many values of x occur
// such that x >= k and x <= k + r
std::unordered_map <int, int> leftCounts;
 
for (auto a : allCounts)
for (int i = 0; i <= r; ++i)
if ((long long)a.first + i > INT_MAX)
break;
else // if (allCounts.find(a.first + i) != allCounts.end())
leftCounts[a.first] += allCounts[a.first + i];
 
// The maximum value of leftCounts is the result
int Max = 0;
for (auto x : leftCounts)
Max = std::max(Max, x.second);
 
return Max;
}
Nitin Kumar <nknitintyagi53@gmail.com>: Nov 22 08:16AM -0800

Title: Jr. C/C++ Developer
Location: Nashville, TN
Contract:6-12 months
 
OPT'S are also fine here
 
 
Skills: AutoSys, C, C++ and Production Support
 
Healthcare experience will be huge plus
 
Thanks and Regards,
 
Nitin Kumar
Professional Recruiter
Saxon Global Inc.
p: (972) 550-9346 ext:365 d: (972) 499-6325
a: 1320 Greenway Drive, Suite #660, Irving, TX
w: www.SaxonGlobal.com e: nitin.k@saxonglobal.com
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Nov 22 04:13PM

On Fri, 22 Nov 2019 16:03:12 +0100
 
> Okay I guessed that much, but why can't the compiler do the latter when
> there is only one pair of braces? The error message clearly indicates
> that an initializer list is considered.
 
That's because in the absence of an initializer-list constructor for
Foo, the compiler is obliged to treat 'Foo f{4,5}' as a call to a
constructor taking two arguments, and you haven't provided an initializer
list constructor for Foo. The only constructors available are the
user-provided constructor (which is a combined default constructor and
conversion constructor), and the compiler provided copy constructor and
move constructor. You will see that the compiler did consider these as
candidates, and then (obviously) found no match.
 
Leaving that aside, I think the main aim is to write intelligible code,
not code which scratches at the margins of C++'s implicit conversions.
So instead of writing 'Foo f{{4,5}}', write 'Foo f{std::vector<int>{4,5}}',
and likewise with your vector of string. Or provide an initializer-list
constructor. You will then be able to understand your intentions when
you come back to it 6 months later.
 
> figured out. So the two string element case fails only because the
> compiler picks the wrong constructor. I find that odd because the
> initializer list seems to be the more obvious choice…
 
You have received a separate answer to that.
James Kuyper <jameskuyper@alumni.caltech.edu>: Nov 22 11:08AM -0500

On 11/18/19 6:21 PM, Sam wrote:
...
> But, if I happen to stumble across something response-worthy and I choose to
> respond, then I will. Especially if the resulting back-and-forth promises to
> be quite entertaining. ...
 
To me, it's just annoying, not entertaining.
 
> irritates someone else, as a result, this is unfortunate; but they are free
> to make their own decisions (and also threaten to ban me from offending
> their delicate eyeballs, as well, if it makes them feel better).
 
I won't threaten to ban you. I'll either do it, or not do it, and if I
do decide to do it, I won't bother announcing that fact, either. I don't
expect people to be afraid of being in my killfile, so I don't see that
I have any leverage for changing their behavior by threatening to put
them there.
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)