Sunday, September 30, 2018

Digest for comp.lang.c++@googlegroups.com - 19 updates in 6 topics

"Öö Tiib" <ootiib@hot.ee>: Sep 29 06:27PM -0700

> > Have you benchmarked against malloc and free compiled as single-threaded?
 
> I used the latest dlmalloc source (the mother of most if not all
> malloc implementations) which does not have lock enabled by default.
 
That dlmalloc has thread-safety when USE_MALLOC_LOCK is defined and that
must be defined in multi threaded program unless someone wants to use
it narrowly for managing state that fully belongs to single thread
only.
There are no authorities, mothers and fathers in software development
products. Instead lot of people write and rewrite and repair and break
similar algorithms from year to year.
 
 
> An approach is basically have a thread local variables to keep track
> of memory pool including system malloc/free is being used.
> And switch to use specific pool using the flag.
 
But your cookmem does not do that, it is just totally thread-unsafe?
Also its architecture is not designed to carry described thread
local pools. It could be instrumented with locks at best.
 
> process model. I do not want to get into thread vs process
> debate. However, threads are typically lighter weight, and
> lower synchronization costs. Resource managements are also simpler.
 
All we talk about is resource management, particularly memory
management here. Indeed it is simpler with thread-safe allocator.
However with thread-unsafe allocator it will be rather complicated
and error-prone. Someone naively using cookmem in their network service
will possibly open it up to wide array of attack vectors.
 
> it to magically solve the problem. It requires certain architectural
> designs such that memory pool provides the benefits. For typical
> non-server applications, one may never need it.
 
Yes but if global malloc and free are replaced with thread-unsafe
then the tools to control that aspect of architecture are abstracted
away from architect's hands. The fundamental block, the thread-safe
allocator upon what architect can build thread-local pools if needed
is now missing from the landscape.
 
> For a long running server process, how to avoid memory leak,
> fragmentation, etc are challenging issues. CookMem is intended
> to be a part of solutions, rather than the solution by itself.
 
I did not ask about other possible programming problems. I was
particularly interested in under what scenario you consider
your thread-unsafe allocator to be viable as part of solution.
We have plenty of other issues to solve, but those are orthogonal
to thread-safety of allocator.
superduperhengyuan@gmail.com: Sep 29 07:10PM -0700

To Öö Tiib
 
CookMem itself does not provide thread safety at all. In fact, I clearly stated that it does not have locking. Yet, this is not an important feature for memory context like CookMem for the reasons below.
 
Instead of using a single malloc that always locks during allocation / deallocation, it is in fact more efficient that each thread has its own lock-free memory context. If you have specific data needs to be shared as shared / global memory, do the locking yourself in the shared / global memory context. After all, if the usage pattern is that most memory allocated in the local thread are not shared, why pay the extra locking cost?
 
In the end, CookMem is just a tool. It is up to you to find if it is useful.
 
Heng Yuan
"Öö Tiib" <ootiib@hot.ee>: Sep 29 07:54PM -0700

> clearly stated that it does not have locking. Yet, this is not an
> important feature for memory context like CookMem for the reasons
> below.
 
Ok.
 
> yourself in the shared / global memory context. After all, if the
> usage pattern is that most memory allocated in the local thread are
> not shared, why pay the extra locking cost?
 
But does cookmem provide that alternative lock-free thread-specific
memory context? My cursory skimming through it made impression that
no, that it would be tricky to enhance it with the feature. Sure, I
may be wrong, but then can you explain, how?
 
> In the end, CookMem is just a tool. It is up to you to find if it is useful.
 
That is certainly true and so I wanted to understand in what scenarios
you consider cookmem to be actually viable as tool.
superduperhengyuan@gmail.com: Sep 29 09:21PM -0700

On Saturday, September 29, 2018 at 7:55:04 PM UTC-7, Öö Tiib wrote:
> memory context? My cursory skimming through it made impression that
> no, that it would be tricky to enhance it with the feature. Sure, I
> may be wrong, but then can you explain, how?
 
You can make it thread-specific. I do not have any code related to this area since it is not limited to this use.
 
You can also take a look at.
 
https://en.wikipedia.org/wiki/Region-based_memory_management
 
CookMem is basically a memory context implemented using algorithms of dlmalloc.
 
Heng Yuan
"Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid>: Sep 29 10:02PM -0700

> CookMem is a a memory context / allocator written in C++11 that can be easily and quickly cleaned up by freeing the segments,
 
> rather than doing individual frees.
 
Like a region allocator?
 
 
"Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid>: Sep 29 10:12PM -0700

>> other thread? Having such complications would make it error-prone,
>> but still useful as performance optimization.
 
> An approach is basically have a thread local variables to keep track of memory pool including system malloc/free is being used. And switch to use specific pool using the flag.
[...]
 
You want to have the "fast path" be thread local, completely isolated
and free of synchronization, implicit or explicit. The "really slow
path" can try to allocate more memory, or fail. I have a lot of
experience wrt creating these types of allocators. Also, take a look at
the allocator in TBB.
 
https://www.threadingbuildingblocks.org/tutorial-intel-tbb-scalable-memory-allocator
 
Also, how does your algorithm deal with false sharing? This can tear
things apart at the seams wrt performance...
"Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid>: Sep 29 10:23PM -0700


> You can also take a look at.
 
> https://en.wikipedia.org/wiki/Region-based_memory_management
 
> CookMem is basically a memory context implemented using algorithms of dlmalloc.
 
There is a tricky way to create an allocator that uses nothing but
memory on a threads stack. It even allows for a memory M allocated by
thread A to be freed by thread B. It is sync-free in the fast path, and
lock-free in the slow path. I was experimenting around with a tricky way
to get another path that can be wait-free. So the levels went something
like:
______________________
(completely thread local)
fast-path = sync-free
 
(can communicate with another thread)
slow-path = wait-free
snail-path = lock-free
______________________
 
This was a long time ago, 10 years at least.
"Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid>: Sep 30 01:08PM -0700

On 9/29/2018 10:02 PM, Chris M. Thomasson wrote:
>> easily and quickly cleaned up by freeing the segments,
 
>> rather than doing individual frees.
 
> Like a region allocator?
 
Perhaps, something like Reaps might help you out:
 
https://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf
 
The key phrase: "rather than doing individual frees"
 
makes me think of region-like allocation.
 
 
James Kuyper <jameskuyper@alumni.caltech.edu>: Sep 30 09:22AM -0400

On 09/29/2018 03:01 AM, Ralf Goertz wrote:
> Am Fri, 28 Sep 2018 09:28:30 -0700 (PDT)
> schrieb Öö Tiib <ootiib@hot.ee>:
...
 
> Thanks, James. But what difference does that make? If the short operands
> get promoted to int and the result gets demoted to short, can that
> really be different from doing it directly? The program
 
Yes, it can. First of all, expressions that would overflow when using
the smaller type can be perfectly safe wehn evaluated using the larger
type. Overflow is undefined behavior, which is generally a bad thing,
but it is quite common on modern machines for signed integer types to
use 2's complement notation, and for overflow to be handled accordingly
without any other negative consequence, which means that expressions
which would overflow in the smaller type might produce exactly the same
result as if that type had been used. However, C still allows
implementations to use sign-magnitude and one's complement notation, for
which that equivalence would not hold.
 
Non 2's complement signed types are rare. However, being aware of the
integer promotions is important for understanding the behavior of other
expressions which are more problematic. Unsigned types with a maximum
representable value that is <= INT_MAX promote to int. As a result,
expressions that you might otherwise have expected to be resolved using
unsigned math may end up using signed math instead.
Tim Rentsch <txr@alumni.caltech.edu>: Sep 30 10:23AM -0700


> A char type is an integral type. [...] char can be signed or
> unsigned but it is also a distinct type from signed char or
> unsigned char. It has no special status beyond that.
 
Actually it does have one: it is one of only three types (the
other two being unsigned char and std::byte) that are exempt from
type access rules given in section 6.10.
 
(It's interesting that signed char is not on that list, which
in C it is.)
Tim Rentsch <txr@alumni.caltech.edu>: Sep 30 10:58AM -0700

>> definition of the italicized term.)
 
> [...] Why can't [u]int8_t be separate types and not typedef'ed to
> [un]signed char [...]?
 
They can be. They don't have to be, but they can be. (I should
add, certainly in C, and I am pretty sure for C++.)
 
> What is gained by these typedef?
 
They make things easy for lazy implementors.
 
To be fair I should add that allowing the [u]intN_t types to be
typedefs gives a degree of freedom to implementations, and which
could be passed on to developers with a compiler option, if that
were thought to be important. Personally I would favor having
such an option, although it isn't one I would put high on a
priority list.
 
> [Assuming CHAR_BIT == 8] I can't think of any task done with
> [u]int8_t that can't also be done as easily without it.
 
Advantages of unsigned char (the type, not necessarily using that
name) over uint8_t:
 
(1) It's guaranteed to exist;
 
(2) It's guaranteed to allow access to any type of
object; and
 
(3) No #include is needed to use it.
 
Advantages of signed char (the type, not necessarily using that
name) over int8_t:
 
(1) It's guaranteed to exist;
 
(2) (In C, not C++) It's guaranteed to allow access to any
type of object; and
 
(3) No #include is needed to use it.
 
The minimum value for signed char may be -127, whereas the
minimum value for int8_t (if it exists) must be -128. But if
that property is desired, it can be checked staticly using the
value of SCHAR_MIN, and if int8_t exists then signed char will
also have a minimum value of -128.
 
Considering the above I see no reason to ever use the [u]int8_t
types, except in cases where it's important to conform to an
existing interface that uses them.
Tim Rentsch <txr@alumni.caltech.edu>: Sep 30 11:11AM -0700

> a byte, then I have to use 16 or 32 bit types. In some cases 8
> bit is enough but there are so many vectors in that set that I get
> into memory trouble when using 16 bit. [...]
 
If you really need an 8-bit integer type, you might consider
making one yourself as a class (or maybe one of the newfangled
enum's, but I haven't used those much). Here is a sketch:
 
#include <iostream>
 
class U8 {
unsigned char v;
public:
U8() : v( 0 ) {}
U8( unsigned uc ) : v( uc ) {}
operator unsigned(){ return v; }
U8 &operator=( unsigned uc ){ v = uc; return *this; }
unsigned value(){ return v; }
};
 
std::istream &
operator >>( std::istream &in, U8 &u8 ){
unsigned u;
in >> u;
u8 = u;
return in;
}
 
int
main(){
U8 u8;
std::cout << "Hello, world\n";
std::cin >> u8;
std::cout << "Value of u8 is " << u8 << "\n";
std::cout << "sizeof u8 = " << sizeof u8 << "\n";
return 0;
}
 
 
Disclaimer: the code compiles and runs, but I'm not sure I've
made good choices about the conversions or really anything else.
The point is it should be possible to define a type with the
properties that you want, provided of course the compiler being
used is good enough to make the class be a single byte in size.
Richard Damon <Richard@Damon-Family.org>: Sep 30 02:26PM -0400

On 9/30/18 1:58 PM, Tim Rentsch wrote:
 
> Considering the above I see no reason to ever use the [u]int8_t
> types, except in cases where it's important to conform to an
> existing interface that uses them.
 
The main reason I use uint8_t/int8_t is stylistic to indicate that what
it holds in treated as a small number and not something 'character'
related. It also signals that I may be making an implicit assumption
that the machine is 'normal' (twos complement, 8 bit byte) rather than
adding a static test of preprocessor symbols. To me, char is only used
to actually hold text, and unsigned char for text that all values need
to be positive (for text processing function calls) or accessing 'raw'
memory.
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Sep 30 08:00PM +0100

On Sun, 30 Sep 2018 10:23:24 -0700
> type access rules given in section 6.10.
 
> (It's interesting that signed char is not on that list, which
> in C it is.)
 
OK, well if one of three counts as having special status, then another
related one is assigning or otherwise evaluating integrals, pointers and
some other objects which have not been initialized and so have
indeterminate value. In most cases this is undefined behaviour, but in
the case of char, unsigned char and std::byte it just gives you another
indeterminate value.
 
int i; // indeterminate value
int j = i; // undefined behaviour
 
char c; // indeterminate value
char d = c; // indeterminate value
 
There may be other special cases of that kind. But these all relate to
the fact that these types may be used for low level byte access in C++
rather than that they are "characters".
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Sep 30 08:03PM +0100

On Sun, 30 Sep 2018 20:00:52 +0100
 
> There may be other special cases of that kind. But these all relate to
> the fact that these types may be used for low level byte access in C++
> rather than that they are "characters".
 
One correction - this only works with char (as opposed to unsigned char
or std::byte) if, in the implementation in question, char is unsigned.
Tim Rentsch <txr@alumni.caltech.edu>: Sep 30 10:05AM -0700


> I presume APL didn't try to define an operator precedence because
> it has SO many operators no one could agree on an order, let alone
> attempt to remember it.
 
My understanding is APL uses right-to-left grouping for all
operators because its inventor, Ken Iverson, thought that this
"right always preferred" (my phrase, not his) choice is somehow
natural or right. If he had wanted to change that, no discussion
or convincing would have been necessary, because he developed the
original core of (what would later be called) APL by himself.
 
I remember reading somewhere that what evolved into APL was
originally called "Iverson notation", but I don't remember where
that was.
woodbrian77@gmail.com: Sep 30 09:38AM -0700

> decided not to. When I read the title of the article, I
> thought it would be a good title for this thread also...
> better late than never.
 
The C++ Middleware Writer has been available now since
October of 2002. It's been an uphill battle in terms of
convincing people to embrace on-line code generation,
but I'm happy with the progress I've been making.
I'm proud of these programs:
https://github.com/Ebenezer-group/onwards/blob/master/src/cmw/tiers/genz.cc
https://github.com/Ebenezer-group/onwards/blob/master/src/cmw/tiers/cmwA.cc
 
One conclusion, from working on that second program, is I
would like to have an intrusive list available in the standard.

 
If you would indulge me, I'd like to have a little guessing
game as far as what quote I am thinking of in regards to this
milestone. Here are a few hints:
1. It's a famous quote that probably at least 70% of
you are familiar with.
 
2. It's not from the Bible and I don't think I've
mentioned it here previously.
 
I'll give an additional hint with each guess.
 
Thanks for all the fishy and not-so-fishy ideas
over the years on how to improve things.
 
 
Brian
Ebenezer Enterprises - Enjoying programming again.
https://github.com/Ebenezer-group/onwards
Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Sep 30 02:41AM +0100

My first political domain/web-site: https://couldhavehadme.com/
 
/Flibble
 
--
"Suppose it's all true, and you walk up to the pearly gates, and are
confronted by God," Bryne asked on his show The Meaning of Life. "What
will Stephen Fry say to him, her, or it?"
"I'd say, bone cancer in children? What's that about?" Fry replied.
"How dare you? How dare you create a world to which there is such misery
that is not our fault. It's not right, it's utterly, utterly evil."
"Why should I respect a capricious, mean-minded, stupid God who creates a
world that is so full of injustice and pain. That's what I would say."
Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Sep 30 02:41AM +0100

On 29/09/2018 23:54, Rick C. Hodgin wrote:
 
> We've been duped by an enemy working globally against mankind.
 
> https://www.youtube.com/watch?v=zOdpfPmeMIA
 
> 25 minutes to challenge your thinking.
 
You can now no longer even be bothered to attempt to disguise your spam as
something else? Now you just post URLs to random shitty Christian videos?
You are a dick. Just fuck off.
 
/Flibble
 
--
"Suppose it's all true, and you walk up to the pearly gates, and are
confronted by God," Bryne asked on his show The Meaning of Life. "What
will Stephen Fry say to him, her, or it?"
"I'd say, bone cancer in children? What's that about?" Fry replied.
"How dare you? How dare you create a world to which there is such misery
that is not our fault. It's not right, it's utterly, utterly evil."
"Why should I respect a capricious, mean-minded, stupid God who creates a
world that is so full of injustice and pain. That's what I would say."
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)