- Obfuscate Assembler (Remove Function Calls, Unroll Loops) - 4 Updates
- Linker discards redundant .o inside .a - 1 Update
- A question about if constexpr - 9 Updates
- Hard work pays off - 3 Updates
- two's complement idea - 4 Updates
- std::atomic<T>::is_lock_free() - 4 Updates
| David Brown <david.brown@hesbynett.no>: Nov 29 11:21AM +0100 On 28/11/2019 16:58, Frederick Gotham wrote: > I suppose another thing I could do is compile the 3rd party library > with some sort of global setting that every function has the > attribute "always_inline". Obfustication like this does not work. If people have enough reason to want to know what your program does, they will find out. It will take them more time and effort, but they will manage it if they want. But if you simply want to make it harder for casual hackers to know what is going on in the code, don't mess around manually like this. Use link-time optimisation, and -O3 optimisation. You might also like to play with some of the parameters to encourage more inlining, cloning, merging, etc. There are also a few more experimental code optimisation options, or ones that change the language semantics, which can be enabled explicitly (see the gcc documentation for details). And some, like the "align" options, you might want to explicitly disable to make code harder to follow. The compiler can do a far better job of inlining, code-reordering, unrolling, etc., than you can hope to manage. Once your LTO-optimised binary is stripped of all extra symbols, it will be very difficult to follow by examination of the object code. |
| Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Nov 29 04:43PM On 28/11/2019 15:58, Frederick Gotham wrote: > The second program is more complicated. It is linked statically with a 3rd party library that I have the source code for. I could go through the code for the 3rd party library and tag each function with 'always_inline' but I'm looking for a cleaner and handier solution. > If I take the assembler for my second program, or even the binary executable file in ELF format, then I could manipulate the assembler (or machine code) in order to remove all function calls (and also unroll all loops if possible). Is there a tool for doing this? > I suppose another thing I could do is compile the 3rd party library with some sort of global setting that every function has the attribute "always_inline". If you look up the word "obfuscation" in a thesaurus will you find that it is a synonym for the word "futile". You. Are. Doing. It. Wrong. /Flibble -- "Snakes didn't evolve, instead talking snakes with legs changed into snakes." - Rick C. Hodgin "You won't burn in hell. But be nice anyway." – Ricky Gervais "I see Atheists are fighting and killing each other again, over who doesn't believe in any God the most. Oh, no..wait.. that never happens." – Ricky Gervais "Suppose it's all true, and you walk up to the pearly gates, and are confronted by God," Byrne asked on his show The Meaning of Life. "What will Stephen Fry say to him, her, or it?" "I'd say, bone cancer in children? What's that about?" Fry replied. "How dare you? How dare you create a world to which there is such misery that is not our fault. It's not right, it's utterly, utterly evil." "Why should I respect a capricious, mean-minded, stupid God who creates a world that is so full of injustice and pain. That's what I would say." |
| scott@slp53.sl.home (Scott Lurndal): Nov 29 05:18PM >On 28/11/2019 16:58, Frederick Gotham wrote: >Once your LTO-optimised binary is stripped of all extra symbols, it will >be very difficult to follow by examination of the object code. Until you run it under QEMU and get a full instruction/register trace. |
| Frederick Gotham <cauldwell.thomas@gmail.com>: Nov 29 10:58AM -0800 "Obfustication like this does not work" I have been reading articles about binary obfuscation, e.g. presentations at Defcon. My own form of obfuscation goes way further than anything else I've ever seen. If the Iranian government were to pay four hackers $120,000 each per year to figure out what my program is doing, I think they'd take about 5 or 6 weeks. Even if you were to give me the output of my own program, and then ask me to determine the state my program was in when it give that output, I think I would spend maybe 4-8 hours writing the program to do so. I think it would take the very best programmers-cum-hackers a few weeks to figure my program out. |
| Frederick Gotham <cauldwell.thomas@gmail.com>: Nov 29 10:42AM -0800 I have a 3rd party library which is 50 - 100 source files. I compile this library to produce a static shared library. All of the object files are gathered together into an archive (In Linux, this is a ".a" file). I then write my own program which links with this static library. For example: g++ main.cpp -o program -l:libmonkey.a When the GNU linker links my own program with the archive file, it only links with the object files inside the archive which are needed. If I had not created the archive file... If I had simply linked my own program with all of the 50-100 object files, then my program would have linked with every single object file -- possibly resulting in unreachable code in the output binary. So... when the linker links with an archive file, it is smart enough to know which object files are needed, and which are not needed. Is there any way to get a list of which object files are actually needed when a program links with an archive file? If I can get such a list, I can remove the redundant object files from the project, and then go on to tweak the object files that are actually needed. I need to reduce a library of nearly a hundred source files, to about ten source files, and then go through each of these 10 source files and apply __attribute__ to every function. I posted in this group yesterday, and one of the people who replied seemed to think that I was trying to hide that I'm linking with a library licensed under GPL (as it is against the GPL license to do what I am doing if my project isn't open-source). The legal advisors in my company have confirmed that I'm allowed to do what I'm doing with this particular 3rd party library. |
| Daniel <danielaparker@gmail.com>: Nov 28 09:09PM -0800 On Thursday, November 28, 2019 at 4:25:03 AM UTC-5, David Brown wrote: > expression happens to be known at compile time (whether it is > technically a C++ constant or not), then the compiler is doing a poor > job at optimising. Which raises the question, what is the point of the constexpr if statement anyway? Daniel |
| "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com>: Nov 29 07:41AM +0100 On 29.11.2019 06:09, Daniel wrote: >> technically a C++ constant or not), then the compiler is doing a poor >> job at optimising. > Which raises the question, what is the point of the constexpr if statement anyway? Two main points: * It communicates to a human reader that the expression is evaluated at compile time, guaranteed. * In a function template the path not taken in a given instantiation, can contain code that wouldn't compile in that instantiation. Maybe more, but it's late in the day for me. - Alf |
| "Öö Tiib" <ootiib@hot.ee>: Nov 28 11:50PM -0800 On Friday, 29 November 2019 07:09:19 UTC+2, Daniel wrote: > > technically a C++ constant or not), then the compiler is doing a poor > > job at optimising. > Which raises the question, what is the point of the constexpr if statement anyway? In template code it is like super #if but outside of templates it is basically only for to tell to readers, maintainers and compiler that the condition is supposed to be constant expression by original logic and something is perhaps gone bad when it is not anymore. |
| David Brown <david.brown@hesbynett.no>: Nov 29 09:23AM +0100 On 28/11/2019 21:04, Robert Wessel wrote: > On Thu, 28 Nov 2019 11:58:40 +0100, Bo Persson <bo@bo-persson.se> > wrote: <snip> >> Don't know exactly which one makes the difference. :-) > Almost certainly /W4 instead of /Wall. The latter is pretty unusable. I am unlikely to remember all the switches given by Bo (though I will remember that any oddities I see in MSVC may be alleviated by switches). But I will definitely try to remember /W4 instead of /Wall. I guess MSVC takes "/Wall" more literally than gcc "-Wall". |
| David Brown <david.brown@hesbynett.no>: Nov 29 09:30AM +0100 On 29/11/2019 06:09, Daniel wrote: >> job at optimising. > Which raises the question, what is the point of the constexpr if statement anyway? > Daniel The part omitted by the false branch has to be able to compile, but otherwise doesn't affect the code. For example: const bool return_int = true; auto foo(void) { if constexpr(return_int) { return 1; } else { return 1.0; } } The return type of "foo" is determined by the "if constexpr". Without the "constexpr", you would get an inconsistent deduction for the return type. Typically, you will see "if constexpr" in templates, I expect. |
| David Brown <david.brown@hesbynett.no>: Nov 29 09:38AM +0100 On 29/11/2019 08:50, Öö Tiib wrote: > basically only for to tell to readers, maintainers and compiler that > the condition is supposed to be constant expression by original logic > and something is perhaps gone bad when it is not anymore. <https://en.cppreference.com/w/cpp/language/if> One other point is that since the discarded part is discarded, any identifiers it uses do not need to be defined or linked in with the code. |
| Paavo Helde <myfirstname@osa.pri.ee>: Nov 29 10:51AM +0200 On 28.11.2019 11:24, David Brown wrote: > my incomplete knowledge of MSVC command line parameters?) > gcc and clang, on the other hand generates a single "return 1;" result > whether you have "constexpr" or not. And no warnings with "-Wall -Wextra". An interesting example demonstrating the benefits of short string optimization. If the constructor or destructor of an otherwise effectively unused local object (as s above) call non-inlined functions, the optimizer cannot (so easily) remove the whole instance. Indeed, the following code produces ca 30 lines of assembly with gcc 9.2 and -std=c++17 -O3, including calls to std::basic_string::_M_create and operator delete: #include <string> int foo() { constexpr int n = 1; constexpr int m = 10; std::string s = "abcdabcdabcdabcd"; if (n < m || s.empty()) { return 1; } return 0; } MSVC also has short string optimization, but it seems by some reason the optimizer has not understood it can apply that for std::string s = "abcd"; or std::string s("abcd"). Interestingly enough, if one writes the string initialization as std::string s("abcd", 4); then also MSVC is able to optimize it away and produces: int foo(void) PROC ; foo mov eax, 1 ret 0 int foo(void) ENDP ; foo So it appears this example is more about SSO and does not tell much about constexpr. |
| Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Nov 29 04:33PM On 27/11/2019 14:03, Alf P. Steinbach wrote: > It's reasonable here because `n < m` is always true, which is known at compile time, and `||` has short-circuit evaluation, which means that the non-`constexpr` empty checking of the string is not considered. > If you switch the values of `n` and `m` then `if constexpr` can no longer be used, because then, though the result is also then knowable at compile time, it isn't /necessarily/ known at compile time. The compiler needs some smarts to understand that `s.empty()` must be `false` here. > That said, I find the Visual C++ warning about constant conditions to be generally unreasonable. For example, it pops up if you write `while(2+2 == 4)` instead of the slightly more idiomatic `for(;;)`. Apparently it no longer does that for `while(true)`, but I just don't trust it, it's evidently tuned to ignore one special literal case, and that's it. VC++ is correct to warn about while(2+2 == 4) and not for(;;) the later being the only acceptable why to do an endless loop (I don't agree that while(true) is just as good as it is syntactically less clean). The reason why VC++ is correct to warn about while(2+2 == 4) is that one of those 2's could be a typo and a variable was meant instead. /Flibble -- "Snakes didn't evolve, instead talking snakes with legs changed into snakes." - Rick C. Hodgin "You won't burn in hell. But be nice anyway." – Ricky Gervais "I see Atheists are fighting and killing each other again, over who doesn't believe in any God the most. Oh, no..wait.. that never happens." – Ricky Gervais "Suppose it's all true, and you walk up to the pearly gates, and are confronted by God," Byrne asked on his show The Meaning of Life. "What will Stephen Fry say to him, her, or it?" "I'd say, bone cancer in children? What's that about?" Fry replied. "How dare you? How dare you create a world to which there is such misery that is not our fault. It's not right, it's utterly, utterly evil." "Why should I respect a capricious, mean-minded, stupid God who creates a world that is so full of injustice and pain. That's what I would say." |
| Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Nov 29 04:35PM On 28/11/2019 07:16, Öö Tiib wrote: > case and leave it on only in performance-analyzing builds but > revisit that decision after every release. What is reasonable > now may become backwards few years later. Utter tosh. if constexpr is far more than a "performance optimization"; it is especially useful in template code. /Flibble -- "Snakes didn't evolve, instead talking snakes with legs changed into snakes." - Rick C. Hodgin "You won't burn in hell. But be nice anyway." – Ricky Gervais "I see Atheists are fighting and killing each other again, over who doesn't believe in any God the most. Oh, no..wait.. that never happens." – Ricky Gervais "Suppose it's all true, and you walk up to the pearly gates, and are confronted by God," Byrne asked on his show The Meaning of Life. "What will Stephen Fry say to him, her, or it?" "I'd say, bone cancer in children? What's that about?" Fry replied. "How dare you? How dare you create a world to which there is such misery that is not our fault. It's not right, it's utterly, utterly evil." "Why should I respect a capricious, mean-minded, stupid God who creates a world that is so full of injustice and pain. That's what I would say." |
| woodbrian77@gmail.com: Nov 28 04:54PM -0800 On Thursday, November 28, 2019 at 5:03:02 PM UTC-6, Öö Tiib wrote: > > https://github.com/Ebenezer-group/onwards > > https://github.com/Ebenezer-group/onwards/blob/master/src/cmw/tiers/genz.cc > Strange little piece of code. What it does? It's the front tier of my code generator. It sends a message to the middle tier and waits for a reply. > in main()? > Also I would manually inline the "leave" since its name > and all the missing comments hide that it was a failure. Leave is called in a number of places. Not sure of a better name. > to read like: > if(auto ind=buf.reserveBytes(1);!buf.receive(ind,marshalFile(mdlFile,buf))) > receive(buf,mdlFile,1); That line is tough, but at least the variable's scope is limited. > ambassador destructively constructing itself into that > "config"? Or why else conversion constructor has a mutable > parameter? That constructor used to be over 100 lines. It's the biggest function by far in the program. I used to wonder if that function would ever get better. Wrestling with it has paid off. I think the parameter used to be const. I'll consider making it const again. > downloading or cloning to get something to work with no any external > aid needed. Here it is building some kind of 4 executables that are > needed to be registered somewhere even to try. Only 2 executables are needed. They are the ones I linked to above. The other two are example programs. I'm happy to give demos of the software. The first step is to download the repo and build it. Yes, after that, you need a little help from me. The example programs can be run without an account. Brian Ebenezer Enterprises https://github.com/Ebenezer-group/onwards |
| "Öö Tiib" <ootiib@hot.ee>: Nov 28 05:20PM -0800 > > Also I would manually inline the "leave" since its name > > and all the missing comments hide that it was a failure. > Leave is called in a number of places. Not sure of a better name. English has plenty of verbs (like retiring, abandoning or ceasing) that have more solemn implications than just leaving. <https://www.thesaurus.com/browse/relinquish> |
| woodbrian77@gmail.com: Nov 29 06:45AM -0800 > > if(auto ind=buf.reserveBytes(1);!buf.receive(ind,marshalFile(mdlFile,buf))) > > receive(buf,mdlFile,1); > That line is tough, but at least the variable's scope is limited. After thinking about it, I changed the second line to this: receiveNull(buf,mdlFile); and changed the interface/implementation of the function I had been calling in that snippet. Thanks for your comments. Brian Ebenezer Enterprises https://github.com/Ebenezer-group/onwards |
| "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com>: Nov 29 12:14AM +0100 On 28.11.2019 21:19, Tim Rentsch wrote: >> teams that manage to hire members with godly hands macros are >> fine. :D > [snip] here, the shortcomings [of macros] don't apply? They do. For example, regarding what was mentioned about scope, you generally can't have two libraries that both define `ONESCOMPLEMENT`. Perhaps library A defines it as above but library B defines it to produce a one's complement. Perhaps both define it as above but using library B you get a redefinition warning. And for example, regarding the so far not mentioned visual impact, the shouting uppercase is an annoying eyesore. - Alf |
| David Brown <david.brown@hesbynett.no>: Nov 29 09:15AM +0100 On 29/11/2019 00:14, Alf P. Steinbach wrote: > you get a redefinition warning. > And for example, regarding the so far not mentioned visual impact, the > shouting uppercase is an annoying eyesore. To be fair, you don't have to use all-caps for macros. Personally, I only use all-caps for macros if the fact that it is a macro is important. So I would write: #define noOfWotsits 10 just as I would write enum { noOfWotists = 10 }; static const int noOfWotsits = 10; const int noOfWotsits = 10; // C++ only depending on the language (C, C++, or compatible with both) and what I want to be able to do with it. Disregarding scoping, these will all have the same effect in almost all sensible uses - I see no earthly reason to insist that one version should be written in all-caps and the others in your favourite multi-word style (snake, camel, etc.). But if I had written this macro (which I would not do - it's just an example): #define SQUARE(x) ((x) * (X)) then I would use all-caps, because it is important to understand that it is a macro. |
| Bo Persson <bo@bo-persson.se>: Nov 29 10:58AM +0100 On 2019-11-29 at 09:15, David Brown wrote: > #define SQUARE(x) ((x) * (X)) > then I would use all-caps, because it is important to understand that it > is a macro. The problem is still that macros defies the use of scopes and namespaces. Unlike you are one of the "members with godly hands" referred to above, a slight mistake might break the next version of the standard library. Note that the appendix "Index of library names" has grown from 35 to 70 pages between C++14 and C++20. None of these may be #define'ed. Just try #define general "Patton" to break floating-point formatting, or #define Init 0 to get odd compile errors from most I/O, or try #define a 'a' to disable <random> numbers. And the library reserved names use all versions of lower_case, UPPER_CASE, and Mixed. Bo Persson |
| David Brown <david.brown@hesbynett.no>: Nov 29 12:49PM +0100 On 29/11/2019 10:58, Bo Persson wrote: >> then I would use all-caps, because it is important to understand that it >> is a macro. > The problem is still that macros defies the use of scopes and namespaces. They do - and when considering "what is the clearest, simplest, most maintainable, most efficient choice" for a fixed value or re-usable sequence of code or characters, then that counts against macros. Whether these count as "problems" or not will depend on the code, and where they are defined. Macros within an implementation file are rarely an issue for scoping - if they are, your implementation file is probably too big, or you have too little imagination in naming your identifiers. But macros in headers can have wider implications. > Unlike you are one of the "members with godly hands" referred to above, > a slight mistake might break the next version of the standard library. That's a risk, certainly. I am a big fan of specifying exactly what standard I want for my code - I don't change versions of the standard library willy-nilly. There are many things that can be badly affected, not just because of macros. > to disable <random> numbers. > And the library reserved names use all versions of lower_case, > UPPER_CASE, and Mixed. Perhaps you misunderstood my post. I didn't say "use lots of macros", or "macros are risk-free", or "macros are better than const". I simply said that when I choose to use a macro, I don't give it a shouty name unless I really need it to shout "I am a macro". Your points are all good reasons to prefer something other than macros (templates, const, inline functions, etc.) where practical, and I don't disagree with any of them. |
| Tim Rentsch <tr.17687@z991.linuxsc.com>: Nov 28 03:39PM -0800 > All objects *must* have complete types in C++ program: > "A program is ill-formed if the definition of any object gives > the object an incomplete type." Not counting the misspelling of proper, there is nothing nonsensical about it. Furthermore although the C++ standard doesn't use those exact words, it does imply that the statement is true. Incidentally, I think the word "bald" doesn't mean what you think it means. The statement "the sun is shining today" could accurately, even if perhaps confusingly, be termed a "bald statement". |
| Tim Rentsch <tr.17687@z991.linuxsc.com>: Nov 28 03:44PM -0800 > as the type of a complete object and when it is used as the type of > a subobject. > ... No it cannot. Refer to n4659, section 6.11, paragraph 1. Note in particular the last sentence, which says in part (with my emphasis added): An object type imposes an alignment requirement on EVERY OBJECT of that type There is no provision for the alignment varying according to whether the object is a complete object or a subobject. |
| "Öö Tiib" <ootiib@hot.ee>: Nov 28 04:33PM -0800 On Friday, 29 November 2019 01:39:26 UTC+2, Tim Rentsch wrote: > Not counting the misspelling of proper, there is nothing nonsensical > about it. Furthermore although the C++ standard doesn't use those > exact words, it does imply that the statement is true. Nonsensical is to assert that standard says something specially about properties of objects of complete types when these are the only kind of objects that may exist in well-formed program. If you mean (leaving the nonsense aside) that "all objects must have proper alignment" is said by standard in some other wording then feel free to quote, I have found it not. > Incidentally, I think the word "bald" doesn't mean what you think it > means. The statement "the sun is shining today" could accurately, > even if perhaps confusingly, be termed a "bald statement". I used "bald assertion" in sense of "a statement used without proof or evidence of truth". Wikipedia claims it is used in that sense about marketing statements ... I think Bonita uses such statements just to post something. |
| "Öö Tiib" <ootiib@hot.ee>: Nov 28 04:43PM -0800 On Friday, 29 November 2019 01:45:04 UTC+2, Tim Rentsch wrote: > OBJECT of that type > There is no provision for the alignment varying according to > whether the object is a complete object or a subobject. Very next paragraph #2 says that: "The alignment required for a type might be different when it is used as the type of a complete object and when it is used as the type of a subobject." |
| You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment