Wednesday, April 18, 2018

Digest for comp.lang.c++@googlegroups.com - 25 updates in 3 topics

wyniijj@gmail.com: Apr 17 09:43PM -0700

Chris Vine於 2018年4月17日星期二 UTC+8下午6時30分00秒寫道:
> alternative to type pun through a union and rely on gcc's and clang's
> language extension which allows this.
 
> Chris
 
You are right, thanks a lots. I'm astonished complier now is so smart!
These is_valid(..) functions are for checking for long sequence of
'valid' 7-bits characters. By your suggestion, I reimplemented
is_valid2(..), and tested (these functions reside in library, not inline).
 
bool is_valid3(const char* data)
{
uint64_t tt;
::memcpy(&tt,data,sizeof(tt));
return !(tt&0x8080808080808080);
};
 
Compiler: g++ v7.3.1
CPU : Intel i5-6500
 
g++ test.cpp
Case1 (all valid) : avg=1.2975 faster (than is_valid(..))
case2 (8 inavalid, 1 invalid): avg=0.5086 faster (slower)
 
g++ test.cpp -O2
Case1 (all valid) : avg=1.8934 faster
Case2 (8 invalid, 1 invalid): avg=1.0723 faster
 
------------------------------------------------------------
And I also tested 6-bytes version of is_valid(..)
 
bool is_valid6(const char* data) // should be free of endian issue
{
uint64_t tt(0x8080808080808080);
::memcpy(&tt,data,6);
return !(tt&0x8080808080808080);
};
 
g++ test.cpp
Case1 (all valid) : avg=0.47313 faster (slower)
case2 (8 inavalid, 1 invalid): avg=0.22621 faster (slower)
 
g++ test.cpp -O2
Case1 (all valid) : avg=1.8695 faster
Case2 (8 invalid, 1 invalid): avg=1.0085 faster
 
[Conclusion] memcpy implement should compile with -O2 to gain
advantage. I can't see union of significance.
wyniijj@gmail.com: Apr 17 10:04PM -0700

Barry Schwarz於 2018年4月18日星期三 UTC+8上午12時35分17秒寫道:
> normal text like ABCD1234 would fail either test.
 
> --
> Remove del for email
 
Thanks reminded me of precedence issue and the IBM machines.
The 7-bits encoding characters are stored in disk and tx/rx
through sockets. I can't figure out how to deal with EBCDIC coding.
wyniijj@gmail.com: Apr 17 10:45PM -0700

Paavo Helde於 2018年4月17日星期二 UTC+8下午6時29分45秒寫道:
> return std::find_if(data, data+8,
> [](char c) {return c&'\x80';})==data+8;
> }
 
Quite some features added since C++11 seemed unsurely necessary to me
I have been busying figuring C++ before C++11 till now.
In this case, I'm not sure if this lambda expression can beat memcpy.
 
In general, learning newer C++ features means one has to spend more
time finding 'bugs' than solving realistic problems (If one studies
or pratices C++ deeply enough)
Juha Nieminen <nospam@thanks.invalid>: Apr 18 07:05AM

> {
> return *reinterpret_cast<const uint64_t*>(data)&0x8080808080808080L;
> };
 
If the pointer is not aligned to a 64-bit integer boundary, it will cause
a hardware interrupt to be triggered in some architectures (which in
practice means the program will crash). With more permissive architectures
it will probably cause a slowdown (of some clock cycles).
 
The standard library does all kinds of optimizations similar to that one
all the time, but it always makes sure that pointers are correctly aligned.
(For example, many std::strcmp() implementations will compare more than one
byte at a time, but they do it in a manner that ensures alignment of the
compared values. In other words, they first compare byte-by-byte until the
first proper alignment offset, then they compare word-by-word, for a given
word size, and then they again compare byte-by-byte for any possible
remaining data.)
Paavo Helde <myfirstname@osa.pri.ee>: Apr 18 12:13PM +0300


> Quite some features added since C++11 seemed unsurely necessary to me
> I have been busying figuring C++ before C++11 till now.
> In this case, I'm not sure if this lambda expression can beat memcpy.
 
Depends on what are your goals, you did no state them in the original post.
 
If you want maximum speed, memcpy is most probably faster than find_if
as it has 8 times less branching in the code.
 
If you want short code and less bugs then find_if is better IMO.
 
 
> In general, learning newer C++ features means one has to spend more
> time finding 'bugs' than solving realistic problems (If one studies
> or pratices C++ deeply enough)
 
C++11 features are meant for reducing bugs, not for creating them. You
can do many things in much less code, meaning there is a lesser chance
to create bugs. Also, if you misuse something you often get compile-time
errors, unlike for memcpy() and friends.
wyniijj@gmail.com: Apr 18 06:00AM -0700

Paavo Helde於 2018年4月18日星期三 UTC+8下午5時13分29秒寫道:
 
> If you want maximum speed, memcpy is most probably faster than find_if
> as it has 8 times less branching in the code.
 
> If you want short code and less bugs then find_if is better IMO.
 
Understood. I did not meant to deny your reply. And sorry I made some typos
 
> can do many things in much less code, meaning there is a lesser chance
> to create bugs. Also, if you misuse something you often get compile-time
> errors, unlike for memcpy() and friends.
 
These words means lots of things to me. But off-topic of this post, think twice.
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 06:20AM -0700

> {
> return *reinterpret_cast<const uint64_t*>(data)&0x8080808080808080L;
> };
 
I would simplify the source code for those who come after you so it
is more visually clear which operation is taking place and where. The
compiler will optimize away the extra declared local variable here:
 
bool is_valid2(const char* data)
{
unit64_t d;

d = *reinterpret_cast<const uint64_t*>(data);
return(d & 0x8080808080808080L);
};
 
It is interchangeable on every platform I'm aware of because all bytes
are addressed identically across the full 8-byte range. Endianness
would not be a factor.
 
I wonder also if you don't actually want this code to test all bits,
rather than just some bits being on:
 
// Test if all bits are on
bool is_valid2(const char* data)
{
uint64_t d;
 
d = *reinterpret_cast<const uint64_t*>(data);
return((d & 0x8080808080808080L) == 0x8080808080808080L);
};
 
??
 
--
Rick C. Hodgin
wyniijj@gmail.com: Apr 18 06:52AM -0700

Juha Nieminen於 2018年4月18日星期三 UTC+8下午3時05分39秒寫道:
> a hardware interrupt to be triggered in some architectures (which in
> practice means the program will crash). With more permissive architectures
> it will probably cause a slowdown (of some clock cycles).
 
Thanks for the information. I know Intel x86 compatible CPU should be
OK and C++ standard says not OK. But C++'s words implies nearly all
kinds of CPU. In my cases, I just worry about CPU that can address say
2-Gbytes of space, and see if there is chance is_valid2(..) can work.
wyniijj@gmail.com: Apr 18 07:44AM -0700

Rick C. Hodgin於 2018年4月18日星期三 UTC+8下午9時20分41秒寫道:
 
> It is interchangeable on every platform I'm aware of because all bytes
> are addressed identically across the full 8-byte range. Endianness
> would not be a factor.
 
I don't understand if my original is_valid2(..) won't work while your
version works (on platforms you are aware of)?
 
 
> ??
 
> --
> Rick C. Hodgin
 
No, I just need to check all character<0x80 (casted to unsigned),
something like checking for valid MIDI or ASCII data bytes.
 
And I felt '0x8080808080808080L' should be written as 0x8080808080808080
,no trailing L(or LL), let compiler decide.
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 10:59AM -0400

> Rick C. Hodgin於 2018年4月18日星期三 UTC+8下午9時20分41秒寫道:
> I don't understand if my original is_valid2(..) won't work while your
> version works (on platforms you are aware of)?
 
Yours will work. It is just unclear in source code because it's doing
all of those things on a single line. The operation is more clear when
you first break out the integer portion to its on variable, then perform
the & operation on that integer.
 
It's just for readability.
 
--
Rick C. Hodgin
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 18 04:02PM +0100

On Wed, 18 Apr 2018 06:20:27 -0700 (PDT)
 
> d = *reinterpret_cast<const uint64_t*>(data);
> return((d & 0x8080808080808080L) == 0x8080808080808080L);
> };
 
Garbage. It's undefined behaviour.
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 11:27AM -0400

On 4/18/2018 11:02 AM, Chris Vine wrote:
>> are addressed identically across the full 8-byte range. Endianness
>> would not be a factor.
 
> Garbage. It's undefined behaviour.
 
Only if you use architectures which do not allow such things. A
simple unit test would confirm if it works or not on any architec-
ture.
 
Compilers often allow things the standard does not. It's not a
guarantee it will work, but it's easy enough to validate.
 
Also, nice to know you actually read my posts. :-) I learn a lot
from you. You're very precise and technical.
 
--
Rick C. Hodgin
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 18 05:03PM +0100

On Wed, 18 Apr 2018 11:27:52 -0400
 
> Only if you use architectures which do not allow such things. A
> simple unit test would confirm if it works or not on any architec-
> ture.
 
Wrong. It's undefined behaviour irrespective of the architecture.
It's about the compiler having the license to optimize the code in
reliance on the strict aliasing rule (which is included in the C and
C++ standards for optimization purposes), so potentially producing the
wrong result in this case. (It might also be undefined behaviour
because of architecture dependent alignment issues, but that's beside
the point: behaviour doesn't become defined by being doubly undefined.)
 
> Compilers often allow things the standard does not. It's not a
> guarantee it will work, but it's easy enough to validate.
 
So, update your compiler or change your optimization level and your code
ceases to work? What a load of horse manure.
 
There seem to be two possible explanations of your explanation.
 
First, you made a mistake, which anyone can do.
 
Or second, you are ignorant of the strict aliasing rule, which is
simple. Namely, that you cannot access an object via a pointer or
reference unless the latter's type is pointer or reference to the
dynamic type of the object, subject to the exceptions in §3.10/10 of
the C++11/14 standard (§6.10/10 of the C++17 standard)[1].
 
Since the OP said that the input form pointed to by 'data' was a
"character sequence", yours is a classic example of how not to do it.
 
> Also, nice to know you actually read my posts. :-)
 
I saw the response to your response and thought your advice needed to
be contradicted.
 
Chris
 
[1] The principle and most useful exception being that you can access
any object or part of an object through a pointer to char or unsigned
char or (since C++17) std::byte, and carry out pointer arithmetic in
doing so.
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 12:21PM -0400

On 4/18/2018 12:03 PM, Chris Vine wrote:
> wrong result in this case. (It might also be undefined behaviour
> because of architecture dependent alignment issues, but that's beside
> the point: behaviour doesn't become defined by being doubly undefined.)
 
I understand that. That's why I say it's easily tested with a unit
test case.
 
>> guarantee it will work, but it's easy enough to validate.
 
> So, update your compiler or change your optimization level and your code
> ceases to work? What a load of horse manure.
 
No. Use the code. Compile it. If it fails the test case, update
a #define setting and recompile to use the alternative mechanism.
 
The code the user is using above will be a couple instructions on a
64-bit machine. To iterate through a loop as you suggest is slower
and more complex.
 
If the simpler version can be used in your compiler, why not use it?
 
> the C++11/14 standard (§6.10/10 of the C++17 standard)[1].
 
> Since the OP said that the input form pointed to by 'data' was a
> "character sequence", yours is a classic example of how not to do it.
 
It was option 3).
 
The algorithm he proposed tested the data in 8-byte chunks at a time.
If it's on a 64-bit machine it's a trivial test. It's even reasonably
trivial on a 32-bit machine.
 
It comes down to looking at what's happening at the machine level, as
opposed to looking at what the language allows.
 
Even in cases of undefined behavior, a startup test to see if something
will work will prevent it from computing improperly in the case where a
compiler optimized away some UB to produce results other than desired.
 
I look at the fundamental operation and acknowledge that, per the OP's
statement that data will always point to 8 or more bytes, that it is
the same operation on that data block.
 
One thing CAlive seeks to do is remove such limitations. CAlive allows
data to be processed as data, and not through type constraints. Each
time a type is brought to bear upon a piece of data, it assumes it is
known to be correct as per the developer's knowledge. If it is in some
way incorrect, then, and only then, will it produce UB, and it will be
solely the direct result of the data, and not the code being changed.
 
I think it's the better philosophy regarding data. It is after all...
data.
 
>> Also, nice to know you actually read my posts. :-)
 
> I saw the response to your response and thought your advice needed to
> be contradicted.
 
Thank you for leaping in to rescue the OP from my advice. I still
stand by my suggestion. The iterative loop method you propose is
slower and very likely completely unnecessary given the nature of
data computing in assembly / machine code.
 
> any object or part of an object through a pointer to char or unsigned
> char or (since C++17) std::byte, and carry out pointer arithmetic in
> doing so.
 
See. Very precise and technical. I like that.
 
--
Rick C. Hodgin
boltar@cylonHQ.com: Apr 18 01:44PM

Hi
 
I'm using a C library, one function of which returns a char* pointer to
memory allocated using malloc(). Is there a way of initialising a standard
string to use this memory directly instead of doing its own allocation and
copy? There answer seems to be no as I can't find anything about it on
stackoverflow or general googling but I wondered if anyone here knows any
better?
 
Thanks for any info
wyniijj@gmail.com: Apr 18 08:01AM -0700

bol...@cylonhq.com於 2018年4月18日星期三 UTC+8下午9時44分39秒寫道:
> stackoverflow or general googling but I wondered if anyone here knows any
> better?
 
> Thanks for any info
 
IIRC, Bjarne Stroustrup sayed something about this: C++ doesn't prevent
one shoot ones own foot.
boltar@cylonHQ.com: Apr 18 03:06PM

On Wed, 18 Apr 2018 08:01:32 -0700 (PDT)
>> Thanks for any info
 
>IIRC, Bjarne Stroustrup sayed something about this: C++ doesn't prevent
>one shoot ones own foot.
 
Thanks, very helpful. Do you have any more nuggets of wisdom from your box of
C++ fortune cookies or is that it?
Sam <sam@email-scan.com>: Apr 18 11:20AM -0400

> copy? There answer seems to be no as I can't find anything about it on
> stackoverflow or general googling but I wondered if anyone here knows any
> better?
 
The short answer is: no there isn't. You can't find anything because no such
thing exists.
 
It's a near certainty that your actual problem is not about finding a way to
use preallocated memory for std::string's storage. You're just assuming that
the solution for your actual problem is to have std::string use preallocated
storage, and that's what you're trying to figure out. Perhaps if a
description of your real problem is provided, you might get a suggestion for
an alternative approach that solves it, without using a std::string.
Paavo Helde <myfirstname@osa.pri.ee>: Apr 18 06:26PM +0300

> copy? There answer seems to be no as I can't find anything about it on
> stackoverflow or general googling but I wondered if anyone here knows any
> better?
 
In C++17 there is std::string_view which does not copy the bytes. But
then you need to remember to free the memory by yourself afterwards,
which is a nuisance. Maybe some custom class combining a string_view and
a deleter? Seems a bit like an overkill.
 
In short, I suggest to just use std::string and forget about the cost of
this copy, until the unlikely case it shows up in the profiler.
Premature optimization is the root of all evil. If your strings are
shorter than a megabyte there should be no problem.
boltar@cylonHQ.com: Apr 18 03:40PM

On Wed, 18 Apr 2018 11:20:09 -0400
>thing exists.
 
>It's a near certainty that your actual problem is not about finding a way to
>use preallocated memory for std::string's storage. You're just assuming that
 
Isn't it? Oh ok, thanks for your insight.
 
>storage, and that's what you're trying to figure out. Perhaps if a
>description of your real problem is provided, you might get a suggestion for
>an alternative approach that solves it, without using a std::string.
 
I could write the entire application in C if I had to. Oddly enough I prefer to
use C++ as it makes life a bit easier, a string class being part of it.
boltar@cylonHQ.com: Apr 18 03:42PM

On Wed, 18 Apr 2018 18:26:14 +0300
>this copy, until the unlikely case it shows up in the profiler.
>Premature optimization is the root of all evil. If your strings are
>shorter than a megabyte there should be no problem.
 
Problem is the function is called and the data assigned to a string for each
row loaded from a database. Those small inefficiencies add up. However it
looks like either I put up with them or just use C string handling functions.
A pity.
Marcel Mueller <news.5.maazl@spamgourmet.org>: Apr 18 05:50PM +0200

> copy? There answer seems to be no as I can't find anything about it on
> stackoverflow or general googling but I wondered if anyone here knows any
> better?
 
Indeed, std::string is by definition mutable and therefore cannot
support immutable memory as backend.
It cannot support memory allocated with the /C/ function malloc either
since it uses C++ allocation. But even C++ new char[] won't help since
the allocation strategy is implementation defined and some string
implementations hold small strings in the instance without external
allocation while larger strings are allocated separately.
 
As work around you can go the other way around and use std::string as
allocator, i.e. malloc replacement, and modify the storage afterwards.
AFAIK there is no undefined behavior in this pattern.
 
If you intend to deal with string /constants/ then C++17's string_view
come into play (as already mentioned).
 
In fact I never used any of the options in production code. Either I
have my completely own string implementation that exactly fit my needs
or I simply deal with std::string not concerning myself with details
like that.
 
 
Marcel
Paavo Helde <myfirstname@osa.pri.ee>: Apr 18 07:19PM +0300

>> shorter than a megabyte there should be no problem.
 
> Problem is the function is called and the data assigned to a string for each
> row loaded from a database. Those small inefficiencies add up.
 
They may add up, but as you know everything is relative and in context,
and in this case you have a database operation which is directly
associated and is done *always* together with std::strings construction.
If constructing the needed std::strings takes less than 5% of your C
library function call you can just forget about it as there is nothing
to win.
 
BTW, if these strings are short, there is a fair chance that std::string
uses SSO with no dynamic memory allocation. OTOH there is probbaly a
malloc()+free() for each string in the C library. If this is the case
then constructing a std::string might have negligible cost compared to
even this malloc()/free() overhead, not to speak about the database
operation.
 
So, don't guess, measure. Run your app in the profiler and see what pops
up. I have found very surprising things this way.
pedro1492@lycos.com: Apr 18 04:35AM -0700

I have some code of the form:
 
C::C( int a, int b, int c): X(a), Y(b), Z(c) {
....
}
 
Sometimes it is invoked
new C(i,j,1)
but others
new C(m,n)
 
So what happens with the latter?
final element Z(c) is undefined?
"James R. Kuyper" <jameskuyper@verizon.net>: Apr 18 09:04AM -0400

> new C(m,n)
 
> So what happens with the latter?
> final element Z(c) is undefined?
 
In order for the second constructor call to be compilable, either
there's a declaration for the constructor that is in scope which
provides a default value for the third parameter, or there's a
declaration in scope for a different constructor that takes only two
arguments.
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.

No comments: