Sunday, April 25, 2021

Digest for comp.lang.c++@googlegroups.com - 13 updates in 2 topics

kegs@provalid.com (Kent Dickey): Apr 25 12:04AM -0500

In article <s5vd9u$1um1$1@gioia.aioe.org>,
>>>> I explained to you why the code doesn't bother to free the
>>>> memorysince it is used for the whole time that the editor is running.
>>>> Freeing it just before exiting the program makes no sense.
 
[ snip]
 
>I know exactly what you are talking about. However, please clean up
>after yourself? Pretty please? UNIX is not the only system out there.
>This is a bad habit to get into. Clean up, gosh darn it.
 
First: Programs do need to erase any temp files and free any shared
resources.
 
I wouldn't have really cared about matching malloc()/free() as an issue, but
I have a problem with a program trying to free() all of its memory when
qutting that is causing a problem itself.
 
1) It allocates shared memory segments. If they already exist, an error is
printed which does not in any way explain the problem.
2) It does millions of malloc()'s (or their equivalent, like many programs now,
it's written in at least 4 languages).
3) I try to exit the program.
4) While walking it's crazy internal structures, to simply call free(), it
finds an inconsistency and immediately exits.
5) It forgets to free the shared memory segments since that would be done
after the consistency check.
 
If you try to re-run the program now, it may fail due to the shared memory
segments still existing.
 
The problem is at no point does the code really care about these internal
consistency checks--except when it wants to exit. And it's deemed not a
real problem since the program does exit (with an error code).
 
If the code simply freed the shared memory segments and did exit(0), and
didn't try to balance malloc() and free(), it would just work.
 
There's a great deal of complexity unwinding complex malloc() chains, and
why bother debugging that mess? Yes, memory leaks are bad, but allocations
that will never be freed are not a problem. Every OS (even trivial ones)
can handle programs not calling free() when the program exits.
 
Kent
"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Apr 24 10:29PM -0700

On 4/24/2021 10:04 PM, Kent Dickey wrote:
> real problem since the program does exit (with an error code).
 
> If the code simply freed the shared memory segments and did exit(0), and
> didn't try to balance malloc() and free(), it would just work.
Actually, I am quite fond of region allocators, where there is no need
to balance between allocating and flushing a region. I wrote one a while
back:
 
https://groups.google.com/g/comp.lang.c/c/7oaJFWKVCTw/m/sSWYU9BUS_QJ
 
https://pastebin.com/raw/f37a23918
 
I used it in various ways, even partitioning it to allow for "partial"
region resets. This is using a nasty hack, to get alignment. But, shit
happens. ;^o
 
Layering a malloc/free on top of a region allocator can sometimes allow
for the free function to be a no-op.
 
 
> why bother debugging that mess? Yes, memory leaks are bad, but allocations
> that will never be freed are not a problem. Every OS (even trivial ones)
> can handle programs not calling free() when the program exits.
 
Still, if you are running any type of "event" loop, memory leaks can be
a problem. I had to debug a horrible one where a subtle race-condition
caused memory leaks over time. The horrible nature of it was the race
condition would only trip at basically random times.
 
Speaking of random, for fun, check out the random number generator thing
I wrote that is based on race conditions:
 
https://groups.google.com/g/comp.lang.c++/c/7u_rLgQe86k/m/XiqELOEECAAJ
 
lol. ;^)
"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Apr 24 10:59PM -0700

On 4/24/2021 9:33 AM, Scott Lurndal wrote:
 
> Cutler "borrowed" most of the NT I/O and Memory subsystems from
> VMS (the VAX operating system). Which had the same issues with
> the non-paged pool.
 
Yeah. It was scary, in a sense. The problem rose its ugly head when to
many IOCP requests were in flight at the same time. The non-paged pool
could get nuked. I definitely had timeout logic to abort a certain class
of connections during times of stress. If a connection is allowed to
persist with no activity forever, it is still there, using up non-paged
memory. So, I would abort them based on timeouts, kind of a "its been
ten minutes, are you still there?" type of thing. I would cancel all of
the timeout pending connections when the server went into a certian
"panic" mode.
 
Way back, I even created a version that spawned clients, and forced them
to make a shitload of connections, and transfer huge files back and
forth, like ping pong. For each established connection the server would
store the current number of connections, along with other system data,
into a file. The system was allowed to crash. Reboot, and the file was
read... System crash at this many connections. So, this gave me a bit of
an insight into how many connections the system could handle before it
eats shit.
"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Apr 24 11:05PM -0700

On 4/24/2021 10:04 PM, Kent Dickey wrote:
> why bother debugging that mess? Yes, memory leaks are bad, but allocations
> that will never be freed are not a problem. Every OS (even trivial ones)
> can handle programs not calling free() when the program exits.
 
Have you ever messed around with Reaps?
 
https://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf
Paavo Helde <myfirstname@osa.pri.ee>: Apr 25 10:57AM +0300

25.04.2021 08:04 Kent Dickey kirjutas:
> real problem since the program does exit (with an error code).
 
> If the code simply freed the shared memory segments and did exit(0), and
> didn't try to balance malloc() and free(), it would just work.
 
We have no way to ensure your diagnosis is correct. The program is
clearly buggy, trying to ignore those bugs won't make the bugs
disappear, it only hides them. In my world, a hidden bug is much worse
than a surfacing bug, as it might reappear at any moment in another place.
 
Based on my experience, I would also say that errors on shutdown are
much more likely caused by bugs in joining of threads or bugs in
shutting down independent components, NOT by freeing dynamically
allocated memory, which is a relatively simple task. Avoiding memory
freeing will most likely not fix those bugs, so one still cannot be
really sure that all the important cleanup like releasing shared memory
is done properly.
 
Technically, if the program is unable or unwilling to join all its
threads properly on shutdown, the correct way to terminate is to call
_exit(), not exit(). The latter will still try to release static data
structures, which may cause havoc with the still running rampant threads.
"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>: Apr 25 01:18AM -0700

On 4/25/2021 12:57 AM, Paavo Helde wrote:
> threads properly on shutdown, the correct way to terminate is to call
> _exit(), not exit(). The latter will still try to release static data
> structures, which may cause havoc with the still running rampant threads.
 
This is one reason why I always feared detached threads, in a sense... ;^o
Paavo Helde <myfirstname@osa.pri.ee>: Apr 25 11:28AM +0300

25.04.2021 11:18 Chris M. Thomasson kirjutas:
>> _exit(), not exit(). The latter will still try to release static data
>> structures, which may cause havoc with the still running rampant threads.
 
> This is one reason why I always feared detached threads, in a sense... ;^o
 
And rightly so!
"Öö Tiib" <ootiib@hot.ee>: Apr 25 06:19AM -0700

On Saturday, 24 April 2021 at 08:06:00 UTC+3, Ian Collins wrote:
> > banking, AI, VR,..., I saw the need for C++ engineer shrinks very fast these years.
> > Most companies are trying to reduce cost by shrinking existing C++ codes to minimal.
> > Fortran, Cobol are still active and high paid for 'high performance' need.
 
I do not see that point ... What companies want to reduce C++ by using more COBOL,
FORTRAN, Ada, D or C? Can anyone point at such companies? Any cite?

> We have the opposite problem, our ever expanding core product is C++
> with no viable alternatives. With our borders basically shut, we are
> also finding it hard to get staff...
 
I also experience same even with borders being open. Wherever there are some real
sensors / actuators into real world to develop there are only those few alternatives
and next to nothing can be done with that Python or JavaScript. So how to reduce
C++ code there?
 
Current main issue is that people want remote work when there is pandemic but for
systems expensive to transport, consisting of number of optional components or
expensive peripherals that have part of control manual it is quite painful to set up.
So the developers have to be physically present at least part of the time. Budgets
available and used are absurd but still the products lag behind because of lack of C++
developers. Writing parts in C can only relieve it very slightly as availability of potent
developers of C is even worse or those are often very same persons fluent in both
but preferring C++.
 
> > To be more on topic, to say C++ is better over C, support from hardware is a must.
> They are pretty much equal except at the extreme low end, that is
> targets not supported by gcc.
 
Yes, and where we have targets supported by gcc there it does not matter at all
that gcc adds technically Fortran, Ada and D as alternatives as companies do
not want to move to those.
wij <wyniijj@gmail.com>: Apr 25 10:23AM -0700

On Sunday, 25 April 2021 at 21:19:34 UTC+8, Öö Tiib wrote:
> > > Fortran, Cobol are still active and high paid for 'high performance' need.
> I do not see that point ... What companies want to reduce C++ by using more COBOL,
> FORTRAN, Ada, D or C? Can anyone point at such companies? Any cite?
 
That saying is based on my own investigation of local job market several years ago.
And, for the question of this Conversation thread, I checked the software engineer
need of the market in my country again for just about 10 pages (no company wants
C++ programer).
I do not remember exactly which company, the one that I remember requires
FORTRAN/C skill is a company or institute for high speed computation (clustered,
large computers). Surely, COBOL is from some banking company, you can also
assume they also need 'high performance'.
Probably a little surprise to me is gaming companies seemingly began to focus
on using 'engines'.
I myself have some career with gaming and security surveillance.., firmware and
hardware. From the experience of hardware development, cost-reduction is a
tough issue. Basically or in such experiences, truly real tech. innovation means
lower cost, vise versa.
 
Q: ...By using *more* COBOL, FORTRAN, Ada, D or C?
A: The reason to using more FORTRAN and C is reasonable as it occurred.
I can only imagine what such a company really want is mathematical skill and
do not want more other restrictions and introducing unnecessary complications.
As to using more COBOL codes, I do not really know, but you can come up with
some reasonable scenario.
As to Ada or D, none in my country I know requires such skill.
James Kuyper <jameskuyper@alumni.caltech.edu>: Apr 25 03:02PM -0400

25.04.2021 11:18 Chris M. Thomasson kirjutas:
...
> This is one reason why I always feared detached threads, in a sense... ;^o
 
I couldn't help thinking how weird that statement would sound outside of
a computer programming context. I imagined someone shying away in terror
when someone else used scissors to cut off a piece of thread. :-)
"Öö Tiib" <ootiib@hot.ee>: Apr 25 12:46PM -0700

> And, for the question of this Conversation thread, I checked the software engineer
> need of the market in my country again for just about 10 pages (no company wants
> C++ programer).
 
Huh? Few look at those ads that all places are full of. These do not indicate who
wants to reduce C++ in any manner. <https://stackoverflow.com/jobs/companies?q=c%2B%2B>
321 results.
 
Job market is nontransparent. Our recruiters periodically ask every single C++-capable
individual whom we want in our country personally if they are interested in new work or
when it is student then even first work. Also they check globally who wants to come into
our little and relatively cold country. They have helped to form migration papers from
places like Russia, Belarus, Argentina, Ecuador and Philippines. Sure the list of people
whom we do not want is lot longer. There are no point to announce anything as it is
more likely that someone whom we don't want will apply. It feels unrelated to
what I was asking.
 
> As to using more COBOL codes, I do not really know, but you can come up with
> some reasonable scenario.
> As to Ada or D, none in my country I know requires such skill.
 
New projects can be started in any language.
Where gcc is compiler there these start in C++ far more frequently than in
FORTRAN or C. But that wasn't even a question as I was particularly asking who
wants to migrate/translate/switch from C++ product/code base to any of those
now and why as I know outright totally none cases.
MrSpook_Xh_gU9dm@pdj82ouz7mi_c.gov.uk: Apr 25 03:18PM

On Fri, 23 Apr 2021 15:05:48 -0300
>and callback is very common. Also, it's very simple and more flexible
>than callbacks based on virtual functions, here is a small contrived
>example:
 
I'm confused why you think the 2nd example is clearer or neater, requiring
as it does a 2nd class instead of handling everything in 1. But everyone has a
prefered style I suppose, its just a matter of taste.
"Öö Tiib" <ootiib@hot.ee>: Apr 25 10:07AM -0700

> I'm confused why you think the 2nd example is clearer or neater, requiring
> as it does a 2nd class instead of handling everything in 1. But everyone has a
> prefered style I suppose, its just a matter of taste.
 
Yes some like to have certain structure of their programs where different
responsibilities are split between classes. Others like to have long blobs of
unstructured code vomits (kind of BASIC-programs) and classes are
inconvenience for them. They are usually embittered and confused why
so few people like their exquisite brand of "taste".
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.

No comments: