Thursday, January 26, 2023

Digest for comp.lang.c++@googlegroups.com - 22 updates in 3 topics

Frederick Virchanza Gotham <cauldwell.thomas@gmail.com>: Jan 26 08:50AM -0800

On Linux x86_64, I've taken the code for an opensource executable program, and I've changed the final linker command to add "-fPIC -shared" so that I get a dynamic shared library instead of an executable file.
 
So I take this ".so" file and I load it into an executable program:
 
void *h = dlopen("libSomeProgram.so", RTLD_NOW);
 
int (*prog_main)(int,char**) = dlsym(h,"main");
 
And so now I can start this other program by executing its 'main' function as follows:
 
char *args[] = { "prog", "-v", "-k", "-c", nullptr, nullptr, nullptr };
prog_main(4, args);
 
I've already built this and tested it and it works fine.
 
At the beginning of all this, I had the choice of building the original program as either a static shared library or a dynamic shared library. I went with dynamic because I didn't want to be burdened with the library's start-up code (i.e. whatever happens inside '_init') until I actually needed the functionality of the library (i.e. until the point that 'dlopen' is called).
 
But then I thought to myself, I can just check what the dynamic shared library does when it's loaded. First of all, I used 'readelf' on the ".so" file, and here's what I saw:
 
(INIT) 0x3000
 
So when the library gets loaded with "dlopen", it starts executing code at the address 0x3000. So next I used 'objdump' to see what's located at address 0x3000:
 
endbr64
sub rsp, 8
mov rax, cs:__gmon_start__ptr
test rax, rax
jz short locationA
call rax ; __gmon_start__
locationA:
add rsp, 8
retn
 
So in C++ code, this basically just does:
 
int init(void)
{
if ( gmon_start_ptr ) return gmon_start_ptr();
return 0;
}
 
So next I checked what the function '__gmon_start__' function does, however there is no such function to be found inside the library. It's actually a 'weak' symbol. I used "nm -D" at the command line, and here's what I see:
 
w __gmon_start__
 
By the way I linked the shared library with "-Wl,--no-undefined" so there should be no unresolved symbols, but undefined weak symbols are allowed.
 
So I think what's happening here is that my shared library is seeking a function called '__gmon_start__' inside the executable file that it's loaded into. I've seen this technique before. For example let's say I were to write a library that optionally prints debug info, well I could have a weak symbol called "void debug_print(char const*)" inside my library, and then the executable file can provide that function to my library if it wants to.
 
Given that the dynamic shared library I built was entirely a C program, I don't have to worry about the constructors of global objects being invoked when 'dlopen' is called. However there are still other ways that code could get executed, for example there might be a global variable that gets initialised with the return value of a function:
 
int some_global_variable = SomeFunction();
 
To get to the bottom line on this though, it seems that my dynamic shared libary doesn't do anything when it's loaded except call the function "__gmon_start__". So if I were to build it as a static shared library and link it at compile time, then my executable program shouldn't be burdened with any extra start-up code. Am I right here? Am I missing something?
 
I need to look into why this executable file looks for "__gmon_start__" in the first place, because the shared library wasn't build with "-pg". It seems that GNU g++ places this weak symbol inside its ELF files even if you don't specify "-pg" at the command line. I don't know why it does that.
scott@slp53.sl.home (Scott Lurndal): Jan 26 05:21PM

>};
> prog_main(4, args);
 
>I've already built this and tested it and it works fine.
 
It is generally not considered to be a good idea
to include a function called 'main' in a dynamic shared object
as it must invariably conflict with the same symbol in the
executable loading the dynamic shared object using dlopen()
which has its own "main" function.
 
 
> if ( gmon_start_ptr ) return gmon_start_ptr();
> return 0;
> }
 
<snip>
>the first place, because the shared library wasn't build with "-pg". It see=
>ms that GNU g++ places this weak symbol inside its ELF files even if you do=
>n't specify "-pg" at the command line. I don't know why it does that.
 
__gmon_start_ptr__ (and a few other static global symbols) are typically provided
by the crt (C-runtime) that gets linked in with the executable. Those symbols
may not be "linked" to the share object when it is loaded, and thus dlopen
may fail. In any case, if you call any of the crt startup functions when
the dynamic object is loaded, you'll conflict with the crt startup functions
that were executed when your program that dlopens the shared object started
(you'll note from objdump that the executable entry point is _start, not main, and _start
calls a bunch of library initialization functions before invoking main).
 
Which is another reason to not use 'main' as the dynamic symbol, or
include any run-time intialization (other than static class initializers) in
the shared object.
Frederick Virchanza Gotham <cauldwell.thomas@gmail.com>: Jan 26 09:37AM -0800

On Thursday, January 26, 2023 at 5:21:17 PM UTC, Scott Lurndal wrote:
> as it must invariably conflict with the same symbol in the
> executable loading the dynamic shared object using dlopen()
> which has its own "main" function.
 
 
This of course will matter when I make a static library because the linker will tell me I have a multiple definition error. But when it comes to dynamic libraries on Linux, you can have more than one function with the same name, and you can even get the other function's address with dlopen(RTLD_NEXT, "main").
 
 
> that were executed when your program that dlopens the shared object started
> (you'll note from objdump that the executable entry point is _start, not main, and _start
> calls a bunch of library initialization functions before invoking main).
 
 
I'll build the original program as an executable, check what its entry point is, and see what it does before main.
Paavo Helde <eesnimi@osa.pri.ee>: Jan 26 07:43PM +0200

26.01.2023 18:50 Frederick Virchanza Gotham kirjutas:
 
> On Linux x86_64, I've taken the code for an opensource executable program, and I've changed the final linker command to add "-fPIC -shared" so that I get a dynamic shared library instead of an executable file.
 
-fPIC is a compiler option, so in general it's not enough to add it only
to the linker command.
 
[...]
 
> At the beginning of all this, I had the choice of building the original program > as either a static shared library
 
What does "static shared library" mean? I guess you mean "static library".
 
or a dynamic shared library. I went with dynamic because I didn't want
to be burdened with the library's start-up code (i.e. whatever happens
inside '_init') until I actually needed the functionality of the library
(i.e. until the point that 'dlopen' is called).
 
I think you said this is open-source program, so why don't you just look
in the source to see if there is any expensive static initialization
taking place. If it does, change the code to perform this initialization
only on demand, and add an initialization function to be called from
your program to trigger this.
 
An why do you care? Does the program startup take 10 seconds more with
the extra library linked in? If not, why do you care?
Frederick Virchanza Gotham <cauldwell.thomas@gmail.com>: Jan 26 03:07PM -0800

On Thursday, January 26, 2023 at 5:43:51 PM UTC, Paavo Helde wrote:
 
> your program to trigger this.
 
> An why do you care? Does the program startup take 10 seconds more with
> the extra library linked in? If not, why do you care?
 
 
I have forked two opensource projects on Github and I'm amalgamating them together.
 
With regard to the program which I want to turn into a static library, well I've gathered all of the ".a" files it needs and I've unzipped them and then combined them with the object files of the main program, and made one big ".a" file out of them, and so now I have one big file called SomeProgram.a.
 
Now if I write a new program and get it to link with SomeProgram.a, I might get a few 'multiple definition' errors, such as 'main' and 'options'.
 
So first I got a list of all the exported symbols in all the object files: find -iname "*.o" | nm -i -r -n1 "{}" | grep -Ev "( U )|( W )|( w )" | cut -d ' ' -f3- | sort | uniq > all_symbols.txt
 
Next I made a command line argument list from them: cat all_symbols.txt | awk '{print "--redefine-sym " $s "=SomeProgram_" $s}' | tr '\n' ' ' > cmd_line_args.txt
 
Next I renamed all of the symbols in all of the object files: find -iname "*.o" | xargs -i -r -n1 objcopy `cat cmd_line_args.txt` "{}"
 
After doing all that, I was assured that there wouldn't be a name collision, so I linked it all together and I didn't get a multiple definition error.
Lynn McGuire <lynnmcguire5@gmail.com>: Jan 25 09:52PM -0600

Hi,
 
I am calling
 
int gettmp (integer *i, integer a, longint *s, longint *temp [100],
integer *fin);
 
with
 
integer a, b, i, j;
longint s[2];
integer ii, ij, fin;
longint temp [2] [100];
 
gettmp (&i, a, s, temp, &fin);
 
But I am getting "Error C2664 'int gettmp(integer *,integer,longint
*,longint *[],integer *)': cannot convert argument 4 from 'longint
[2][100]' to 'longint *[]' chemtran_dll c:\dii\shr\struct\tree.cpp 116 "
from
Visual C++ 2015.
 
And advice here ?
 
Both "longint" and "integer" are defined to be "long long" types.
 
Thanks,
Lynn McGuire
Ben Bacarisse <ben.usenet@bsb.me.uk>: Jan 26 04:04AM


> I am calling
 
> int gettmp (integer *i, integer a, longint *s, longint *temp
> [100], integer *fin);
 
The temp argument has type longint **. The 100 is ignored and the []s
just mean a pointer in this context.
 
> integer ii, ij, fin;
> longint temp [2] [100];
 
> gettmp (&i, a, s, temp, &fin);
 
The type of temp here is longint (*)[100]. It's a pointer to the first
of some (in this case two) arrays of 100 longints.
 
> [2][100]' to 'longint *[]' chemtran_dll c:\dii\shr\struct\tree.cpp 116 " from
> Visual C++ 2015.
 
> And advice here ?
 
The type you need for the parameter is longint (*temp)[100]. You could
write it as longint temp[2][100] if you wanted to, but I wouldn't.
 
--
Ben.
Lynn McGuire <lynnmcguire5@gmail.com>: Jan 26 02:12PM -0600

On 1/25/2023 10:04 PM, Ben Bacarisse wrote:
 
>> And advice here ?
 
> The type you need for the parameter is longint (*temp)[100]. You could
> write it as longint temp[2][100] if you wanted to, but I wouldn't.
 
Thanks ! I changed gettmp to:
 
int gettmp (integer *i, integer a, longint *s, longint temp [][100],
integer *fin);
 
and now it compiles.
 
Sincerely,
Lynn McGuire
scott@slp53.sl.home (Scott Lurndal): Jan 25 11:37PM

>> That's all there is to it.
 
>Which particular message you are referring to that shows there was a
>glitch? Post the FULL HEADER OF THAT MESSAGE.
 
I posted the full set of headers and the body of the message in
question which was extracted from nntp.usenetserver.com using
the pan newsreader yesterday. The headers matched those in David's
screenshot perfectly, including the posting date. The body,
on the other hand was correct (a message from Muttley) on nntp.usenetserver.com
and incorrect (the rant) on eternal-september. A poster further noted later
that eternal-september has explictly acknowledged that there was
a glitch in the association between headers and bodies in their
index, which explains how David (and others) saw the incorrect body
on eternal-september.
 
So, it was a server glitch, propogated when someone replied to the
article on eternal-september quoting the false body and the thread
began.
Spiros Bousbouras <spibou@gmail.com>: Jan 25 11:49PM

On Thu, 26 Jan 2023 00:00:29 +0100
> says it is from Muttley with the subject "Re: Compute Unique Numbers in
> a Set". The message id is "tqb2m7$1aqa$1@gioia.aioe.org". The
> timestamp is 19.01.2023 10:31, and it is 22 lines.
 
I note that <tqb2m7$1aqa$1@gioia.aioe.org> no longer exists on
news.eternal-september.org .It exists on other servers , for example
news.cyber23.de .
Spiros Bousbouras <spibou@gmail.com>: Jan 26 12:09AM

On Wed, 25 Jan 2023 14:52:00 -0500
> > than a server/client glitch.
 
> The time that Bonita posted is the time for the message that Muttley
> posted to comp.lang.c.
 
If you mean <tqb2m7$1aqa$1@gioia.aioe.org> which says
 
Really? How would go about predicting what packets will arrive on the network
or when a user will press a key then?
 
then it was posted on comp.lang.c++ not comp.lang.c .
 
[...]
 
> give him the evidence he saw? Such a hack would, legitimately, qualify
> as a server-glitch, and would therefore not refute his assertion that
> there was such a glitch.
 
I don't think that a hack qualifies as a glitch.
 
--
vlaho.ninja/prog
iKook <iKook@gmail.com>: Jan 26 02:27AM

On 25/01/2023 23:00, David Brown wrote:
 
> I recommend you check these things for yourself.
 
I have done it on MixMin. See below
> "36403165-3cf1-4b73-8ad1-da339b960339n@googlegroups.com", from Ilya
> Shambat, newsgroups rec.arts.books.  Xref:
> reader01.eternal-september.org rec.arts.books:26566.
 
No it's not. See the full message here:
 
 
> >Absolutely not.
 
> Really? How would go about predicting what packets will arrive on the network
> or when a user will press a key then?
 
Is there anything else we could check for ourselves to verify your claims?
James Kuyper <jameskuyper@alumni.caltech.edu>: Jan 25 11:55PM -0500

On 1/25/23 19:09, Spiros Bousbouras wrote:
> On Wed, 25 Jan 2023 14:52:00 -0500
> James Kuyper <jameskuyper@alumni.caltech.edu> wrote:
...
>> as a server-glitch, and would therefore not refute his assertion that
>> there was such a glitch.
 
> I don't think that a hack qualifies as a glitch.
 
The server shouldn't allow such a hack to occur. If it does, the server
has a flaw. Keep in mind that this wasn't just a matter of faking a few
of the headers. Every header from Muttley's message appeared in the
summary header list. Every header from Ilya Shambat's message appeared
in the body of the corresponding message body. Several of those headers
are supposed to be created or modified by the server itself just before
delivering them to the subscriber. Any hack that could affect those
headers implies a large degree of control over at least the news server
daemon, and possibly many other parts of the server computer.
James Kuyper <jameskuyper@alumni.caltech.edu>: Jan 25 11:55PM -0500

On 1/25/23 21:27, iKook wrote:
>> Injection-Info: gioia.aioe.org; logging-data="43850"; posting-host="QImplQW63EVMF2Hp+OxW0A.user.gioia.aioe.org"; mail-complaints-to="abuse@aioe.org";
>> X-Notice: Filtered by postfilter v. 0.9.2
>> Xref: news.mixmin.net comp.lang.c++:206390
 
That's not particularly surprising. If there was indeed a server glitch,
it's entirely plausible that sooner or later the glitch might eventually
get fixed.
 
...
>> Really? How would go about predicting what packets will arrive on the network
>> or when a user will press a key then?
 
> Is there anything else we could check for ourselves to verify your claims?
 
Is there really any need to ask for more information? Do you have any
good reason to think David was lying about what he found?
David Brown <david.brown@hesbynett.no>: Jan 26 09:19AM +0100

On 26/01/2023 00:17, ikook wrote:
>> That's all there is to it.
 
> Which particular message you are referring to that shows there was a
> glitch? Post the FULL HEADER OF THAT MESSAGE.
 
If you are interested, read the other posts in this thread. I've gone
over this in excruciating detail, and I am not doing it again. And if
you've read them all, and followed my guide to replicating the issue,
and /still/ have more questions - then I can try to answer them.
David Brown <david.brown@hesbynett.no>: Jan 26 09:23AM +0100

On 26/01/2023 00:49, Spiros Bousbouras wrote:
 
> I note that <tqb2m7$1aqa$1@gioia.aioe.org> no longer exists on
> news.eternal-september.org .It exists on other servers , for example
> news.cyber23.de .
 
This could be a result of the resynchronisation fix done on the server.
If the overview database and main database are out of sync, and they get
resynchronised, then one of them has to change. But I have no idea of
the details of how that might be done.
 
I doubt if the loss of Muttley's real message will be a big blow to
comp.lang.c++ followers using news.eternal-september.org.
David Brown <david.brown@hesbynett.no>: Jan 26 09:44AM +0100

On 26/01/2023 03:27, iKook wrote:
> On 25/01/2023 23:00, David Brown wrote:
 
>> I recommend you check these things for yourself.
 
> I have done it on MixMin. See below
 
That might be useful for comparison.
 
> Is there anything else we could check for ourselves to verify your claims?
 
Yes - check it on the server that had the glitch!
 
 
If you drive to the car workshop and tell them your car is making a
funny noise, would you be impressed if a whole bunch of mechanics rush
out and look at every other car in the car park? Then when they find
the other cars make no noise, they tell you it must be a troll in your car?
 
How is it that there are people in news.admin.net-abuse.usenet who have
supposedly all this "experience and expertise", and are willing to look
up messages all over the place and draw all sorts of conclusions about
people, articles, and servers - and yet none of them seems willing to
take a look at the server that had the glitch?
 
 
I've been involved in computers a /long/ time. A rare glitch on a
server does not surprise me. What /does/ surprise and frustrate me are
two things.
 
One is that so many people have been so quick to accuse and blame Bonita
of all sorts of things, without any consideration for alternative
explanations of events. That includes people who know her postings
well, and those who had not heard of her until this thread. Neither
groups have any excuse for their judgemental attitude.
 
The other is that so many apparent experts are determined to look
everywhere except where the problem lies. It apparently does not matter
what details I give, what explanations I post, what guides I give to
reproducing the problem so that you can see it for yourself. No,
apparently the "expert" handling is to ignore all I write, deny that
servers can ever have glitches, and ask for more evidence.
 
This really isn't rocket science. If the problem was on /my/ computer,
it would be hard to replicate - but it is on a /free/ Usenet server.
Make a /free/ account there, and look at the message in question. It
should take all of about 10 minutes - and really it only takes one of
you "net-abuse" lot to do it (since you don't trust or believe /me/).
 
If you don't want to do that, fair enough. But please stop asking how
to verify my claims and then ignoring the answer.
"Öö Tiib" <ootiib@hot.ee>: Jan 26 01:40AM -0800

On Thursday, 26 January 2023 at 10:44:48 UTC+2, David Brown wrote:
> explanations of events. That includes people who know her postings
> well, and those who had not heard of her until this thread. Neither
> groups have any excuse for their judgemental attitude.
 
We know BM, who will happily quote whatever garbage kill-filed by many in
its entirety on one hand and alter texts of actual discussion for to misrepresent
others on the other hand. Usenet is indeed getting old and unreliable and that
unfortunately adds more opportunities to misbehave.
 
> reproducing the problem so that you can see it for yourself. No,
> apparently the "expert" handling is to ignore all I write, deny that
> servers can ever have glitches, and ask for more evidence.
 
That was good, made some "experts" to look less "apparent" for me. ;-)
David Brown <david.brown@hesbynett.no>: Jan 26 01:10PM +0100

On 25/01/2023 23:58, Spiros Bousbouras wrote:
> in the very first post you talked about a server glitch. You are a technical
> person , you *know* that for bugs , glitches , etc. one ideally should
> mention precise steps which would allow one to reproduce the problem.
 
If I had tested with Pan at that point, I would have mentioned it. I
have tested more as I investigated further - I have also learned from
other people's posts.
 
> mentioning that there was one specific server on which you saw the issue. I
> was actually wondering what the hell was happening and you weren't giving
> specifics.
 
I have tried to give specifics as people have asked for them, as well as
guidance to reproduce the problem. At the start, however, I had no idea
how widespread the issue might have been - maybe it was temporary, maybe
it was related to the client, maybe it had propagated to all Usenet
servers. When I first saw it, I assumed it was local to my PC - it was
only gradually that I realised Bonita had seen the same glitch I had,
and it was not just on my system.
 
 
>> <https://paste.pics/b4149f38abb4e210da0a71886714d014>
 
> This isn't very helpful and in particular it doesn't help to distinguish
> whether it's a server issue or a Thunderbird issue. Just because Thunderbird
 
The screenshot is not Thunderbird, it is Pan. Thunderbird and Pan
appear to show threads in exactly the same way, as far as I could tell.
 
> "timestamp 19.01.2023, 10:31" .If Thunderbird does not allow you to see the
> header exactly as it received it from the server then it's no good for
> diagnosing issues.
 
Pan and Thunderbird are both perfectly good for diagnosing that there
was a ****ing server glitch that meant the wrong article was returned
for the message id in the header. I didn't know how to make Pan show
the message ID from the header until just recently. But you don't need
the id or other information to see that there is clearly a mismatch.
 
> since this is the server which displays the issue.
 
> Method 1
 
> You need to have the lynx text based browser installed.
 
I haven't used it for years, but I do have it installed.
 
> articles with such numbers or they might refer to different posts. But the
> important point here is that whatever posts the server returns , they must
> have been posted to comp.lang.c++ , possibly among other groups.
 
Yes - assuming everything is working correctly at the server.
 
> From: Ilya Shambat <ibshambat@gmail.com>
 
> .Note that it was not (cross)posted on comp.lang.c++ so it should have
> never been returned.
 
Correct. It should never have been there - it is a result of the glitch.
 
> =====
 
> .This correctly appears in the response by the server since it was crossposted
> on comp.lang.c++ .
 
Yes, I can confirm this is all as you say.
 
> 2.5 .Yes it's an old version but it's the only one I have handy and I haven't
> bothered to learn Python 3.whatever .But the script should run on the newest
> Python with little or no modifications. You run the script with
 
I have Python 2.7, rather than 2.5, on this system. But it works fine
there too. Python was my first thought as a way to do a
newsreader-independent check of the issue (I had no idea that lynx could
handle the NNTP protocol). I hadn't bothered trying it because I had
confirmed the issue with a different newsreader (Pan).
 
But trying your script gives the expected result - the rec.arts.book
article is returned by the server when asking for articles that appear
in the overview data for comp.lang.c++.
 
Thanks for making these recipes and the script, and confirming the glitch.
 
 
 
 
David Brown <david.brown@hesbynett.no>: Jan 26 01:14PM +0100

On 26/01/2023 10:40, Öö Tiib wrote:
> its entirety on one hand and alter texts of actual discussion for to misrepresent
> others on the other hand. Usenet is indeed getting old and unreliable and that
> unfortunately adds more opportunities to misbehave.
 
I don't think anyone who is familiar with Bonita thinks she is a model
Usenet citizen (though she is much more willing to follow standards with
suitable quotations and attributions than she used to be). And there is
no doubt that quoting the entire rambling essay to top-post a single
comment is bad style. But that doesn't mean it is right or fair to
accuse her of things she /didn't/ do. news.eternal-september.org
attributed the essay to Muttley - it was not Bonita's idea.
 
iKook <iKook@gmail.com>: Jan 26 06:13PM

On 26/01/2023 08:44, David Brown wrote:
 
> Yes - check it on the server that had the glitch!
 
I tried but you need to sign up with them. This is not I can do now. We
just have to accept that servers can go wrong. AIOE is also down for the
past 3 or 4 days including their main website.
iKook <iKook@gmail.com>: Jan 26 06:22PM

On 26/01/2023 04:55, James Kuyper wrote:
> Is there really any need to ask for more information? Do you have any
> good reason to think David was lying about what he found?
I don't have reasons to doubt anything, good or bad. I just like to
confirm if this problem has propagated to other servers that I use. I
use MixMin and news.us.Usenet-News.net and both looks OK to me but what
worried me is that the cross-post was to some recreational books
newsgroup. This, IMO, is a recipe to invite spam to a technical
newsgroup such as this one.
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)