- Weird use of maps. - 7 Updates
- Should I use Release-Acquire ordering or Sequentially-consistent ordering to implement spinlock with atomic? - 8 Updates
- ""Rust is the future of systems programming, C is the new Assembly": Intel principal engineer, Josh Triplett" - 1 Update
- What is easiest to use static C++ code analyser? - 5 Updates
- newbie question: exceptions - 1 Update
- Why does this work on Xcode ??? - 1 Update
- Anyone working with Graphiz and DOT (directed graphs) - 1 Update
| James Kuyper <jameskuyper@alumni.caltech.edu>: Aug 28 09:30AM -0400 I had to fix a bug recently in poorly documented code written by someone else, who was not available for consultation. This code is written in C++, but uses Qt, which I'm not as familiar with as I'd like. However, I believe that QMap, QString, and QVariant do not differ from std::map, std::string, and std::variant in ways that are significant to the following explanation - please correct me if I'm wrong about this. I can't show the actual code (it's proprietary), but if you find the following description confusing, I could post equivalent code. The context is a message string that gets broken up into multiple parts for transmission. Each part is transmitted along with a message ID, a part number, and the total number of parts. The parts were not guaranteed to arrive at their destination in any particular order. Parts of multiple different messages might be intermingled with each other. The receiving code stored, for each a message, a map containing the parts of the message received so far, to be recombined after all parts have been received. Here's where it got decidedly odd: the code uses a QMap<QString, QVariant), using the text of each message part as a key, and passing the part number as the corresponding value. When all parts had arrived, it would then sort the map by it's values, and then extract the corresponding keys to reconstruct the original message. The bug was that if one part of a message has the same exact contents as a previously processed part, then the new part number would replace the existing part number, rather than adding a new entry to the map. As a result, the number of parts collected would never match the number originally sent, so the message could never be completed, and eventually it got discarded. In practice, this situation isn't likely to come up; I found the bug during testing only because I was testing to confirm limits on the total message size, so I used a test message containing many repetitions of "1234567890", in order to more easily count how many characters I was sending. As soon as I tested with a message that needed more than 10 parts, a duplicate was guaranteed. The solution was trivial: use a QMap<int,QString> rather than QMap<QString,QVariant>, using the part number as the key. There was absolutely no need in this code to generalize by using QVariant - it was used solely to store int values. This rearrangement had the added advantage of avoiding the slow sort step, because QMap already allows fast extraction of values in key order. The only explanation I could find for this peculiar design was a short comment: "I'd prefer using QMap<int,QString>, which would be faster - but then it wouldn't be a QVariantMap". The string "QVariantMap" occurs nowhere else anywhere in the relevant code base, and there's nothing about it at <https://doc.qt.io/>. I have no idea what he was thinking about when he wrote that comment. Any suggestions? At a higher level in the same code, all of the messages currently being received are stored in a QMap<QString, QVariant>, where the key is the message ID, and value is the QMap containing the corresponding part map. That code does something that also strikes me as inefficient and strange, but at least it's not actually buggy: It first searches for the messageID in the message map. If it is not found, it creates a new part map, inserts the current part into that map, and then inserts that part map into the message map. That code makes sense to me. However, if it does find the messageID in the message map, it copies the corresponding part map into a local variable, inserts the new part into that map, and then copies the part map back into the message map, replacing the old part map for that same message ID. I find this quite strange: it wastes time and memory creating a copy of the part map, it wastes time destroying the old part map when the new part map is copied in it's place, and finally wastes time when the new part map gets destroyed. It seems more sensible to me to insert the new part directly into the part map while it is stored inside the message map. In principle, a compiler might optimize the actual code accordingly, but I wouldn't recommend relying on that. Can anyone suggest any good reason why someone might use the more elaborate approach that was actually chosen? Note that the code does not do anything else with either the message map or the local copy of the part map. While the code as a whole is multi-threaded, everything I described happens within a single thread. The code contains none of the protections it would have to have if the message map were shared between threads. |
| Jens Kallup <jkallup@web.de>: Aug 28 04:22PM +0200 Hello, QMap, and other template class "extend" the corespond stdC++ std::map<k,v> class. So with QMap<QString,int> int_map, can store data by "key" as QString (std::string) with value "int": -> int_map["foo"] = 4; -> int_map["bar"] = 2; under stdC++, you can use "emplace". If you look deeper to Qt, Qt give you more compact coding. The fact, that QString based from std::string , you can convert std::string to QString to std::string, there are members like: QString s1; std::string s2 = s1.toStdString(); char buffer[] = s2.c_str(); in Qt, you have cheaper worries about converting the text encoding (ASCII -> UTF-8 <- ASCII) If you would like transmit some data aka Client/Server software, you should look to the RFC's descripted internet protocolls. So you can define your own "static" data Header like: ID | 4 Bytes (as int) CMD | 4 Bytes (as char[4] for "send", or "quit"... BODY <n> | n Bytes, where n is the number of bytes that | comes after sending the header. It is easy to create a QtSocket stream, to send data. And Qt comes with a SSL layer, so you don't have to code abstrakt layer with the original ssl .so/.dll. And, you could create your own/self signed certificates for your SSL stream. The only thing, that may be new to Qt beginner's are the Slots, and Signal handling methods for various events that can be occur/fired by the programmer/user. Hope This Helps Jens |
| "Öö Tiib" <ootiib@hot.ee>: Aug 28 09:00AM -0700 On Wednesday, 28 August 2019 16:30:51 UTC+3, James Kuyper wrote: > The context is a message string that gets broken up into multiple parts > for transmission. Each part is transmitted along with a message ID, a > part number, and the total number of parts. How can receiver realize that all parts have been received? Is the message part count known compile time and part numbers sequential and zero-based? Then std::array<std::optional<std::string>,Count> might be simplest. > Can anyone suggest any good reason why > someone might use the more elaborate approach that was actually chosen? It can be anything. For example initially ambitious requirements did change into simpler and so it was cut out from something way more overcomplicated. The file change history from repo (and commit comments) may illuminate it a bit (if such exist). Results of developers with different skill levels differ by several orders of magnitude, but no one notices when that inefficiency is in some non-critical part of program. |
| Mike Terry <news.dead.person.stones@darjeeling.plus.com>: Aug 28 07:19PM +0100 On 28/08/2019 14:30, James Kuyper wrote: > and passing the part number as the corresponding value. When all parts > had arrived, it would then sort the map by it's values, and then extract > the corresponding keys to reconstruct the original message. That sounds so strange, that I wonder if there's a simple explanation: the original coder just confused the ordering of the two map parameters? Perhaps the intention was to have a map by part number. If multiple part numbers were encountered they would be deemed retransmissions and it would be OK to just keep the latest one. If duplicate message part texts are rare, the original (incorrect) code would seem to be working, and after that there could have been a reluctance to change it, and any issues coming up would just be worked around. (Like, "hey, seems I need to sort by values to get things in the right order?". Yes, seems a bit crazy, but maybe the coder was new. I've seen equally crazy code from new coders, or coders forced out of their normal coding environment!) > compiler might optimize the actual code accordingly, but I wouldn't > recommend relying on that. Can anyone suggest any good reason why > someone might use the more elaborate approach that was actually chosen? This sounds again like a new coder - perhaps the coder didn't realise it was possible to retrieve a /reference/ to the part map? Maybe the coder was searching for how to retrieve a value from a map, and the first function encountered was a "retrieve by value" function. Under time pressure perhaps the thought was, "ok I can make it work using this...". Or perhaps the coder saw how to retrieve a reference, but it was a reference to a QVariant, not a part map, and the coder was stuck on this. (Put another way, maybe the part map in message map either was, or seemed (to the coder) to effectively be a const entry that could only be updated through replacement, not modification...) Mike. |
| usenet@stegropa.de (Stefan Große Pawig): Aug 28 08:22PM +0200 > nowhere else anywhere in the relevant code base, and there's nothing > about it at <https://doc.qt.io/>. I have no idea what he was thinking > about when he wrote that comment. Any suggestions? QVariant has a large set of toXXX() methods to get at the contained type, including possible conversions. Among these is toMap(), which returns a QMap<QString, QVariant>. Is that map of yours stored in another QVariant? If so (now or sometime back in the past), that could explain the requirement to use a QMap<QString, QVariant>. -Stefan |
| Jorgen Grahn <grahn+nntp@snipabacken.se>: Aug 28 07:33PM On Wed, 2019-08-28, James Kuyper wrote: > The bug was that if one part of a message has the same exact contents as > a previously processed part, then the new part number would replace the > existing part number, rather than adding a new entry to the map. That's such an obvious flaw that I'd not trust the rest of that programmers choices to make sense. It's simply not a sane way to do the trivial task of message reassembly. /Jorgen -- // Jorgen Grahn <grahn@ Oo o. . . \X/ snipabacken.se> O o . |
| "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Aug 28 02:54PM -0700 On 8/28/2019 9:00 AM, Öö Tiib wrote: >> for transmission. Each part is transmitted along with a message ID, a >> part number, and the total number of parts. > How can receiver realize that all parts have been received? Think of sending a file using packets. Each one would have an offset into the file, where it physically belongs. So, the sender would send the file size to the receiver, get an ack to make sure it got it, retry if not. Once this is completed, the receiver would create a memory mapped file of the same size it received. The sender blasts packets each containing a file offset and payload. The receiver gets a packet, and writes its payload to the offset in the memory mapped file. The receiver can ask for missing offsets for any dropped, missing packets. Duplicate packets will simply overwrite existing offsets. Here is the idea, I posted as SenderX: https://groups.google.com/forum/#!original/alt.winsock.programming/-QXuP9174Gg/Qt1LB0tPt-8J Afaict, it kind of sounds like Zmodem. |
| Ruki <waruqi@gmail.com>: Aug 27 06:02PM -0700 I want to implement spinlock with atomic. Should I use Release-Acquire ordering or Sequentially-consistent ordering to ensure visibility for multicore? I read the c11 atomic documentation (https://en.cppreference.com/w/c/atomic/memory_order), which only clearly shows the multicore visible to the Sequentially-consistent ordering. |
| David Brown <david.brown@hesbynett.no>: Aug 28 09:15AM +0200 On 28/08/2019 03:02, Ruki wrote: >> instruction on all multi-core systems. This may become a >> performance bottleneck since it forces the affected memory accesses >> to propagate to every core. Acquire and release ordering are usually what you want for locks - /acquire/ the lock, then /release/ it. But why are you implementing spinlocks ? Are you writing your own OS? If not, then usually the OS's own spinlocks will be a better choice. (If this is for learning purposes or fun, then of course that is an excellent reason.) |
| Ruki <waruqi@gmail.com>: Aug 28 12:32AM -0700 在 2019年8月28日星期三 UTC+8下午3:15:32，David Brown写道： > If not, then usually the OS's own spinlocks will be a better choice. > (If this is for learning purposes or fun, then of course that is an > excellent reason.) Not all os provide spinlocks, and like nginx, they also use their own spinlock implementation. |
| Bonita Montero <Bonita.Montero@gmail.com>: Aug 28 10:56AM +0200 Use acquire-semantics when locking, release-semantics when unlocking. |
| "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Aug 28 02:01AM -0700 On 8/27/2019 6:02 PM, Ruki wrote: > I want to implement spinlock with atomic. Should I use Release-Acquire ordering or Sequentially-consistent ordering to ensure visibility for multicore? > I read the c11 atomic documentation (https://en.cppreference.com/w/c/atomic/memory_order), which only clearly shows the multicore visible to the Sequentially-consistent ordering. >> Total sequential ordering requires a full memory fence CPU instruction on all multi-core systems. This may become a performance bottleneck since it forces the affected memory accesses to propagate to every core. What type of spinlock are you going for? A Dekker? |
| "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Aug 28 02:05AM -0700 On 8/28/2019 1:56 AM, Bonita Montero wrote: > Use acquire-semantics when locking, release-semantics when unlocking. Some special spinlocks require sequential consistency. Even an x86 cannot handle a store followed by a load to another location. Some locks depend on that. IBM's hazard pointers do. |
| Bonita Montero <Bonita.Montero@gmail.com>: Aug 28 11:16AM +0200 > Some special spinlocks require sequential consistency. Even an x86 > cannot handle a store followed by a load to another location. Some > locks depend on that. IBM's hazard pointers do. I think he is talking about a usual mutex. |
| "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Aug 28 02:20PM -0700 On 8/28/2019 2:16 AM, Bonita Montero wrote: >> cannot handle a store followed by a load to another location. Some >> locks depend on that. IBM's hazard pointers do. > I think he is talking about a usual mutex. Well, then acquire/release works. But its not sufficient for certain things. The usual pattern goes: ___________________________ LOCK(); // Atomically gain mutual exclusion ACQUIRE_MEMBAR(); // critical section RELEASE_MEMBAR(); UNLOCK(); // Atomically release mutual exclusion ___________________________ Now, some spinlocks need a membar in the LOCK() logic itself. And can avoid using the acquire. Something like: ___________________________ LOCK_WITH_MEMBAR(); // Gain mutual exclusion, using something special... //ACQUIRE_MEMBAR(); // not needed // critical section RELEASE_MEMBAR(); UNLOCK(); // Atomically release mutual exclusion ___________________________ |
| Lynn McGuire <lynnmcguire5@gmail.com>: Aug 28 04:11PM -0500 ""Rust is the future of systems programming, C is the new Assembly": Intel principal engineer, Josh Triplett" https://hub.packtpub.com/rust-is-the-future-of-systems-programming-c-is-the-new-assembly-intel-principal-engineer-josh-triplett/ "At Open Source Technology Summit (OSTS) 2019, Josh Triplett, a Principal Engineer at Intel gave an insight into what Intel is contributing to bring the most loved language, Rust to full parity with C. In his talk titled Intel and Rust: the Future of Systems Programming, he also spoke about the history of systems programming, how C became the "default" systems programming language, what features of Rust gives it an edge over C, and much more." Lynn |
| Jens Kallup <jkallup@web.de>: Aug 28 03:57PM +0200 Hello, if you would like to write your own text analyze tool(s), I point you to have a look over the compiler-compiler tool "bison", and "flex", and yacc. bison, flex, and yacc are available under GPL licensed Open Source Code for most all operating system's (Linux, Windows, MacOS). Them the tool(s) can be download for C/C++, Pascal/Delphi, Lazarus, etc... Hope This Helps Jens |
| legalize+jeeves@mail.xmission.com (Richard): Aug 28 06:18PM [Please do not mail me a copy of your followup] Jens Kallup <jkallup@web.de> spake the secret code >analyze tool(s), I point you to have a look >over the compiler-compiler tool "bison", >and "flex", and yacc. Sorry, but this is a HORRIBLE idea for parsing C++ code. If you want to write your own code analysis tools, use the clang libraries. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Terminals Wiki <http://terminals-wiki.org> The Computer Graphics Museum <http://computergraphicsmuseum.org> Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com> |
| Jorgen Grahn <grahn+nntp@snipabacken.se>: Aug 28 07:22PM On Wed, 2019-08-28, Richard wrote: > Sorry, but this is a HORRIBLE idea for parsing C++ code. > If you want to write your own code analysis tools, use the clang > libraries. There's also no indication whatsoever that the OP wanted to write the tools himself -- he (understandably) didn't even want to evaluate the ones which already are available to him. /Jorgen -- // Jorgen Grahn <grahn@ Oo o. . . \X/ snipabacken.se> O o . |
| legalize+jeeves@mail.xmission.com (Richard): Aug 28 08:00PM [Please do not mail me a copy of your followup] Jorgen Grahn <grahn+nntp@snipabacken.se> spake the secret code >There's also no indication whatsoever that the OP wanted to write the >tools himself -- he (understandably) didn't even want to evaluate the >ones which already are available to him. I agree, but suggesting that one use lex/yacc for parsing C++ is an approach that many people have tried and failed. It's a really awful idea. That suggestion should not be left out in the open unchallenged. This is particularly true when you can just use the clang libraries to get a perfectly conforming C++20 parser that will just hand you an AST. -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Terminals Wiki <http://terminals-wiki.org> The Computer Graphics Museum <http://computergraphicsmuseum.org> Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com> |
| David Brown <david.brown@hesbynett.no>: Aug 28 10:21PM +0200 On 28/08/2019 22:00, Richard wrote: > This is particularly true when you can just use the clang libraries to > get a perfectly conforming C++20 parser that will just hand you an > AST. Another option that would be interesting for static analysis is a gcc plugin, possibly written in a different language (Python is a reasonable choice for such tasks). I can't tell you whether clang or gcc would be the best starting point for making your own static analysis tools, but I look at both of them /long/ before looking at lex/yacc. |
| red floyd <dont.bother@its.invalid>: Aug 26 10:29AM -0700 On 8/23/2019 11:32 PM, Rosario19 wrote: > is dinamically created and free > and if one doesn't know how many leak his/her progam has > but for remain it seems ok If you design your classes properly and use RAII, you will not have leaks. Also, by using the Standard Library, you should be able to avoid using new/delete directly, again, avoiding leaks. |
| Tim Rentsch <tr.17687@z991.linuxsc.com>: Aug 26 09:53AM -0700 >> very much swimming upstream here: do you really want to say you are >> smarter than all the people who have participated in the C and C++ >> working groups to standardize these languages? I am still in the process of reading through the cascade of downstream messages, but I wanted to respond to this one item and try to get it out of the way. > * The idea that I'm arguing a view at odds with the standard. I'm > not. Check how many in this group agree with your view of UB: that's > 0. The comment of mine quoted above has nothing to do with undefined behavior or your ideas about it. You have misunderstood the context of the comment. > blah. If that were the case then all those non-committee members who > have filed Defect Reports against the standard, must be megalomaniacs > who think they're very very smart. That's just not so. First, I never said you think you're smarter than everyone who has worked on the C or C++ standards. Second, I never said someone who thinks the Standard is flawed in some way (and thus in contradiction to all the committee members who have worked on it) must be wrong about that. You have misheard me. ISTM that often what you hear doesn't match what I said; whether or not it happens often, it did happen in this case. As regards defect reports, several items are worth noting. One, most defect reports are filed by people either who are on the committee or who have spent extensive time working with them. Two, a lot of defect reports are responded to by saying, in essence, "this isn't a problem, the standard is fine." (I don't know what the percentages are, for any of the classes of filers.) Three, most defect reports involve issues that are odd corner cases in one way or another, and probably just got overlooked. In contrast, the notion of observable behavior is a core issue, and undoubtedly one that has been given a lot of attention and serious thought. The people who worked on C89-C99 are by nature a practical, pragmatic bunch, and not particularly given to academic flights of fancy. Given that the notion of observable behavior (not the name, but the ideas behind it) has survived essentially unchanged to the present day, the idea that it is impractical or overly academic in nature is rightly regarded with some skepticism. And that's all I was saying. > * It's a purely social argument. It isn't a social argument; rather it is an argument of technical competence and one reflecting the nature of the people who worked on the early C standards. If anything, your argument that my ideas about undefined behavior must be wrong because no one in comp.lang.c++ agrees with me is a social argument. Even if it were true (which it isn't) that no one here agrees with me, comp.lang.c++ is a social collection of people, as are the subsets that choose to participate in various threads and topics. So if anyone is making a social argument, it's you. |
| Melzzzzz <Melzzzzz@zzzzz.com>: Aug 28 02:43AM > try { p_children = &g_links.at(str_parent); } catch (...) { return; } // The method 'at' will throw if no such key exists in the map You shlouldn't catch this it is fatal error. -- press any key to continue or any other to quit... U ničemu ja ne uživam kao u svom statusu INVALIDA -- Zli Zec Na divljem zapadu i nije bilo tako puno nasilja, upravo zato jer su svi bili naoruzani. -- Mladen Gogala |
| You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment