- alignment and endian issues - 14 Updates
- Can one initialise std::string with pre-allocated memory? - 9 Updates
- parameterized constructor with missing fields? - 2 Updates
wyniijj@gmail.com: Apr 17 09:43PM -0700 Chris Vine於 2018年4月17日星期二 UTC+8下午6時30分00秒寫道: > alternative to type pun through a union and rely on gcc's and clang's > language extension which allows this. > Chris You are right, thanks a lots. I'm astonished complier now is so smart! These is_valid(..) functions are for checking for long sequence of 'valid' 7-bits characters. By your suggestion, I reimplemented is_valid2(..), and tested (these functions reside in library, not inline). bool is_valid3(const char* data) { uint64_t tt; ::memcpy(&tt,data,sizeof(tt)); return !(tt&0x8080808080808080); }; Compiler: g++ v7.3.1 CPU : Intel i5-6500 g++ test.cpp Case1 (all valid) : avg=1.2975 faster (than is_valid(..)) case2 (8 inavalid, 1 invalid): avg=0.5086 faster (slower) g++ test.cpp -O2 Case1 (all valid) : avg=1.8934 faster Case2 (8 invalid, 1 invalid): avg=1.0723 faster ------------------------------------------------------------ And I also tested 6-bytes version of is_valid(..) bool is_valid6(const char* data) // should be free of endian issue { uint64_t tt(0x8080808080808080); ::memcpy(&tt,data,6); return !(tt&0x8080808080808080); }; g++ test.cpp Case1 (all valid) : avg=0.47313 faster (slower) case2 (8 inavalid, 1 invalid): avg=0.22621 faster (slower) g++ test.cpp -O2 Case1 (all valid) : avg=1.8695 faster Case2 (8 invalid, 1 invalid): avg=1.0085 faster [Conclusion] memcpy implement should compile with -O2 to gain advantage. I can't see union of significance. |
wyniijj@gmail.com: Apr 17 10:04PM -0700 Barry Schwarz於 2018年4月18日星期三 UTC+8上午12時35分17秒寫道: > normal text like ABCD1234 would fail either test. > -- > Remove del for email Thanks reminded me of precedence issue and the IBM machines. The 7-bits encoding characters are stored in disk and tx/rx through sockets. I can't figure out how to deal with EBCDIC coding. |
wyniijj@gmail.com: Apr 17 10:45PM -0700 Paavo Helde於 2018年4月17日星期二 UTC+8下午6時29分45秒寫道: > return std::find_if(data, data+8, > [](char c) {return c&'\x80';})==data+8; > } Quite some features added since C++11 seemed unsurely necessary to me I have been busying figuring C++ before C++11 till now. In this case, I'm not sure if this lambda expression can beat memcpy. In general, learning newer C++ features means one has to spend more time finding 'bugs' than solving realistic problems (If one studies or pratices C++ deeply enough) |
Juha Nieminen <nospam@thanks.invalid>: Apr 18 07:05AM > { > return *reinterpret_cast<const uint64_t*>(data)&0x8080808080808080L; > }; If the pointer is not aligned to a 64-bit integer boundary, it will cause a hardware interrupt to be triggered in some architectures (which in practice means the program will crash). With more permissive architectures it will probably cause a slowdown (of some clock cycles). The standard library does all kinds of optimizations similar to that one all the time, but it always makes sure that pointers are correctly aligned. (For example, many std::strcmp() implementations will compare more than one byte at a time, but they do it in a manner that ensures alignment of the compared values. In other words, they first compare byte-by-byte until the first proper alignment offset, then they compare word-by-word, for a given word size, and then they again compare byte-by-byte for any possible remaining data.) |
Paavo Helde <myfirstname@osa.pri.ee>: Apr 18 12:13PM +0300 > Quite some features added since C++11 seemed unsurely necessary to me > I have been busying figuring C++ before C++11 till now. > In this case, I'm not sure if this lambda expression can beat memcpy. Depends on what are your goals, you did no state them in the original post. If you want maximum speed, memcpy is most probably faster than find_if as it has 8 times less branching in the code. If you want short code and less bugs then find_if is better IMO. > In general, learning newer C++ features means one has to spend more > time finding 'bugs' than solving realistic problems (If one studies > or pratices C++ deeply enough) C++11 features are meant for reducing bugs, not for creating them. You can do many things in much less code, meaning there is a lesser chance to create bugs. Also, if you misuse something you often get compile-time errors, unlike for memcpy() and friends. |
wyniijj@gmail.com: Apr 18 06:00AM -0700 Paavo Helde於 2018年4月18日星期三 UTC+8下午5時13分29秒寫道: > If you want maximum speed, memcpy is most probably faster than find_if > as it has 8 times less branching in the code. > If you want short code and less bugs then find_if is better IMO. Understood. I did not meant to deny your reply. And sorry I made some typos > can do many things in much less code, meaning there is a lesser chance > to create bugs. Also, if you misuse something you often get compile-time > errors, unlike for memcpy() and friends. These words means lots of things to me. But off-topic of this post, think twice. |
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 06:20AM -0700 > { > return *reinterpret_cast<const uint64_t*>(data)&0x8080808080808080L; > }; I would simplify the source code for those who come after you so it is more visually clear which operation is taking place and where. The compiler will optimize away the extra declared local variable here: bool is_valid2(const char* data) { unit64_t d; d = *reinterpret_cast<const uint64_t*>(data); return(d & 0x8080808080808080L); }; It is interchangeable on every platform I'm aware of because all bytes are addressed identically across the full 8-byte range. Endianness would not be a factor. I wonder also if you don't actually want this code to test all bits, rather than just some bits being on: // Test if all bits are on bool is_valid2(const char* data) { uint64_t d; d = *reinterpret_cast<const uint64_t*>(data); return((d & 0x8080808080808080L) == 0x8080808080808080L); }; ?? -- Rick C. Hodgin |
wyniijj@gmail.com: Apr 18 06:52AM -0700 Juha Nieminen於 2018年4月18日星期三 UTC+8下午3時05分39秒寫道: > a hardware interrupt to be triggered in some architectures (which in > practice means the program will crash). With more permissive architectures > it will probably cause a slowdown (of some clock cycles). Thanks for the information. I know Intel x86 compatible CPU should be OK and C++ standard says not OK. But C++'s words implies nearly all kinds of CPU. In my cases, I just worry about CPU that can address say 2-Gbytes of space, and see if there is chance is_valid2(..) can work. |
wyniijj@gmail.com: Apr 18 07:44AM -0700 Rick C. Hodgin於 2018年4月18日星期三 UTC+8下午9時20分41秒寫道: > It is interchangeable on every platform I'm aware of because all bytes > are addressed identically across the full 8-byte range. Endianness > would not be a factor. I don't understand if my original is_valid2(..) won't work while your version works (on platforms you are aware of)? > ?? > -- > Rick C. Hodgin No, I just need to check all character<0x80 (casted to unsigned), something like checking for valid MIDI or ASCII data bytes. And I felt '0x8080808080808080L' should be written as 0x8080808080808080 ,no trailing L(or LL), let compiler decide. |
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 10:59AM -0400 > Rick C. Hodgin於 2018年4月18日星期三 UTC+8下午9時20分41秒寫道: > I don't understand if my original is_valid2(..) won't work while your > version works (on platforms you are aware of)? Yours will work. It is just unclear in source code because it's doing all of those things on a single line. The operation is more clear when you first break out the integer portion to its on variable, then perform the & operation on that integer. It's just for readability. -- Rick C. Hodgin |
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 18 04:02PM +0100 On Wed, 18 Apr 2018 06:20:27 -0700 (PDT) > d = *reinterpret_cast<const uint64_t*>(data); > return((d & 0x8080808080808080L) == 0x8080808080808080L); > }; Garbage. It's undefined behaviour. |
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 11:27AM -0400 On 4/18/2018 11:02 AM, Chris Vine wrote: >> are addressed identically across the full 8-byte range. Endianness >> would not be a factor. > Garbage. It's undefined behaviour. Only if you use architectures which do not allow such things. A simple unit test would confirm if it works or not on any architec- ture. Compilers often allow things the standard does not. It's not a guarantee it will work, but it's easy enough to validate. Also, nice to know you actually read my posts. :-) I learn a lot from you. You're very precise and technical. -- Rick C. Hodgin |
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 18 05:03PM +0100 On Wed, 18 Apr 2018 11:27:52 -0400 > Only if you use architectures which do not allow such things. A > simple unit test would confirm if it works or not on any architec- > ture. Wrong. It's undefined behaviour irrespective of the architecture. It's about the compiler having the license to optimize the code in reliance on the strict aliasing rule (which is included in the C and C++ standards for optimization purposes), so potentially producing the wrong result in this case. (It might also be undefined behaviour because of architecture dependent alignment issues, but that's beside the point: behaviour doesn't become defined by being doubly undefined.) > Compilers often allow things the standard does not. It's not a > guarantee it will work, but it's easy enough to validate. So, update your compiler or change your optimization level and your code ceases to work? What a load of horse manure. There seem to be two possible explanations of your explanation. First, you made a mistake, which anyone can do. Or second, you are ignorant of the strict aliasing rule, which is simple. Namely, that you cannot access an object via a pointer or reference unless the latter's type is pointer or reference to the dynamic type of the object, subject to the exceptions in §3.10/10 of the C++11/14 standard (§6.10/10 of the C++17 standard)[1]. Since the OP said that the input form pointed to by 'data' was a "character sequence", yours is a classic example of how not to do it. > Also, nice to know you actually read my posts. :-) I saw the response to your response and thought your advice needed to be contradicted. Chris [1] The principle and most useful exception being that you can access any object or part of an object through a pointer to char or unsigned char or (since C++17) std::byte, and carry out pointer arithmetic in doing so. |
"Rick C. Hodgin" <rick.c.hodgin@gmail.com>: Apr 18 12:21PM -0400 On 4/18/2018 12:03 PM, Chris Vine wrote: > wrong result in this case. (It might also be undefined behaviour > because of architecture dependent alignment issues, but that's beside > the point: behaviour doesn't become defined by being doubly undefined.) I understand that. That's why I say it's easily tested with a unit test case. >> guarantee it will work, but it's easy enough to validate. > So, update your compiler or change your optimization level and your code > ceases to work? What a load of horse manure. No. Use the code. Compile it. If it fails the test case, update a #define setting and recompile to use the alternative mechanism. The code the user is using above will be a couple instructions on a 64-bit machine. To iterate through a loop as you suggest is slower and more complex. If the simpler version can be used in your compiler, why not use it? > the C++11/14 standard (§6.10/10 of the C++17 standard)[1]. > Since the OP said that the input form pointed to by 'data' was a > "character sequence", yours is a classic example of how not to do it. It was option 3). The algorithm he proposed tested the data in 8-byte chunks at a time. If it's on a 64-bit machine it's a trivial test. It's even reasonably trivial on a 32-bit machine. It comes down to looking at what's happening at the machine level, as opposed to looking at what the language allows. Even in cases of undefined behavior, a startup test to see if something will work will prevent it from computing improperly in the case where a compiler optimized away some UB to produce results other than desired. I look at the fundamental operation and acknowledge that, per the OP's statement that data will always point to 8 or more bytes, that it is the same operation on that data block. One thing CAlive seeks to do is remove such limitations. CAlive allows data to be processed as data, and not through type constraints. Each time a type is brought to bear upon a piece of data, it assumes it is known to be correct as per the developer's knowledge. If it is in some way incorrect, then, and only then, will it produce UB, and it will be solely the direct result of the data, and not the code being changed. I think it's the better philosophy regarding data. It is after all... data. >> Also, nice to know you actually read my posts. :-) > I saw the response to your response and thought your advice needed to > be contradicted. Thank you for leaping in to rescue the OP from my advice. I still stand by my suggestion. The iterative loop method you propose is slower and very likely completely unnecessary given the nature of data computing in assembly / machine code. > any object or part of an object through a pointer to char or unsigned > char or (since C++17) std::byte, and carry out pointer arithmetic in > doing so. See. Very precise and technical. I like that. -- Rick C. Hodgin |
boltar@cylonHQ.com: Apr 18 01:44PM Hi I'm using a C library, one function of which returns a char* pointer to memory allocated using malloc(). Is there a way of initialising a standard string to use this memory directly instead of doing its own allocation and copy? There answer seems to be no as I can't find anything about it on stackoverflow or general googling but I wondered if anyone here knows any better? Thanks for any info |
wyniijj@gmail.com: Apr 18 08:01AM -0700 bol...@cylonhq.com於 2018年4月18日星期三 UTC+8下午9時44分39秒寫道: > stackoverflow or general googling but I wondered if anyone here knows any > better? > Thanks for any info IIRC, Bjarne Stroustrup sayed something about this: C++ doesn't prevent one shoot ones own foot. |
boltar@cylonHQ.com: Apr 18 03:06PM On Wed, 18 Apr 2018 08:01:32 -0700 (PDT) >> Thanks for any info >IIRC, Bjarne Stroustrup sayed something about this: C++ doesn't prevent >one shoot ones own foot. Thanks, very helpful. Do you have any more nuggets of wisdom from your box of C++ fortune cookies or is that it? |
Sam <sam@email-scan.com>: Apr 18 11:20AM -0400 > copy? There answer seems to be no as I can't find anything about it on > stackoverflow or general googling but I wondered if anyone here knows any > better? The short answer is: no there isn't. You can't find anything because no such thing exists. It's a near certainty that your actual problem is not about finding a way to use preallocated memory for std::string's storage. You're just assuming that the solution for your actual problem is to have std::string use preallocated storage, and that's what you're trying to figure out. Perhaps if a description of your real problem is provided, you might get a suggestion for an alternative approach that solves it, without using a std::string. |
Paavo Helde <myfirstname@osa.pri.ee>: Apr 18 06:26PM +0300 > copy? There answer seems to be no as I can't find anything about it on > stackoverflow or general googling but I wondered if anyone here knows any > better? In C++17 there is std::string_view which does not copy the bytes. But then you need to remember to free the memory by yourself afterwards, which is a nuisance. Maybe some custom class combining a string_view and a deleter? Seems a bit like an overkill. In short, I suggest to just use std::string and forget about the cost of this copy, until the unlikely case it shows up in the profiler. Premature optimization is the root of all evil. If your strings are shorter than a megabyte there should be no problem. |
boltar@cylonHQ.com: Apr 18 03:40PM On Wed, 18 Apr 2018 11:20:09 -0400 >thing exists. >It's a near certainty that your actual problem is not about finding a way to >use preallocated memory for std::string's storage. You're just assuming that Isn't it? Oh ok, thanks for your insight. >storage, and that's what you're trying to figure out. Perhaps if a >description of your real problem is provided, you might get a suggestion for >an alternative approach that solves it, without using a std::string. I could write the entire application in C if I had to. Oddly enough I prefer to use C++ as it makes life a bit easier, a string class being part of it. |
boltar@cylonHQ.com: Apr 18 03:42PM On Wed, 18 Apr 2018 18:26:14 +0300 >this copy, until the unlikely case it shows up in the profiler. >Premature optimization is the root of all evil. If your strings are >shorter than a megabyte there should be no problem. Problem is the function is called and the data assigned to a string for each row loaded from a database. Those small inefficiencies add up. However it looks like either I put up with them or just use C string handling functions. A pity. |
Marcel Mueller <news.5.maazl@spamgourmet.org>: Apr 18 05:50PM +0200 > copy? There answer seems to be no as I can't find anything about it on > stackoverflow or general googling but I wondered if anyone here knows any > better? Indeed, std::string is by definition mutable and therefore cannot support immutable memory as backend. It cannot support memory allocated with the /C/ function malloc either since it uses C++ allocation. But even C++ new char[] won't help since the allocation strategy is implementation defined and some string implementations hold small strings in the instance without external allocation while larger strings are allocated separately. As work around you can go the other way around and use std::string as allocator, i.e. malloc replacement, and modify the storage afterwards. AFAIK there is no undefined behavior in this pattern. If you intend to deal with string /constants/ then C++17's string_view come into play (as already mentioned). In fact I never used any of the options in production code. Either I have my completely own string implementation that exactly fit my needs or I simply deal with std::string not concerning myself with details like that. Marcel |
Paavo Helde <myfirstname@osa.pri.ee>: Apr 18 07:19PM +0300 >> shorter than a megabyte there should be no problem. > Problem is the function is called and the data assigned to a string for each > row loaded from a database. Those small inefficiencies add up. They may add up, but as you know everything is relative and in context, and in this case you have a database operation which is directly associated and is done *always* together with std::strings construction. If constructing the needed std::strings takes less than 5% of your C library function call you can just forget about it as there is nothing to win. BTW, if these strings are short, there is a fair chance that std::string uses SSO with no dynamic memory allocation. OTOH there is probbaly a malloc()+free() for each string in the C library. If this is the case then constructing a std::string might have negligible cost compared to even this malloc()/free() overhead, not to speak about the database operation. So, don't guess, measure. Run your app in the profiler and see what pops up. I have found very surprising things this way. |
pedro1492@lycos.com: Apr 18 04:35AM -0700 I have some code of the form: C::C( int a, int b, int c): X(a), Y(b), Z(c) { .... } Sometimes it is invoked new C(i,j,1) but others new C(m,n) So what happens with the latter? final element Z(c) is undefined? |
"James R. Kuyper" <jameskuyper@verizon.net>: Apr 18 09:04AM -0400 > new C(m,n) > So what happens with the latter? > final element Z(c) is undefined? In order for the second constructor call to be compilable, either there's a declaration for the constructor that is in scope which provides a default value for the third parameter, or there's a declaration in scope for a different constructor that takes only two arguments. |
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment