Monday, April 3, 2017

Digest for comp.lang.c++@googlegroups.com - 19 updates in 7 topics

"Chris M. Thomasson" <invalid@invalid.invalid>: Apr 03 03:12PM -0700

On 3/20/2017 5:01 PM, Chris M. Thomasson wrote:
 
> The entire example code can be found here:
 
> http://pastebin.com/raw/6nG6t422
 
> Can you run this shi%?
[...]
 
This can be multiplexed wrt using more than one of these stacks and a
little help wrt relaxed atomic loads and stores that try to give a
little meta data on the most recently used container...
"Chris M. Thomasson" <invalid@invalid.invalid>: Apr 02 06:29PM -0700

On 4/1/2017 3:35 PM, Ian Collins wrote:
 
> More mow the idea are becoming mainstream. I appreciate that you have
> been post about lock-free for many years, you'll probably be able to
> converse with people other than your self before too long!
 
Very nice. I am looking forward to this time Ian. Thank you.
 
 
> As a point of reference, there are a small group of us on the large
> project team I am currently working with who are investigation how we
> can incorporate this stuff into the existing code.
 
What type of project? I am interested in any use patterns. Be careful
wrt using some of this stuff because it can make a part of the engine
more powerful. This enhancement will cast its increased performance on
other parts of the system, in the form of more load. Might end up making
a set of locks contend more than they ever had before...
Ian Collins <ian-news@hotmail.com>: Apr 03 05:10PM +1200

On 04/ 3/17 01:29 PM, Chris M. Thomasson wrote:
>> project team I am currently working with who are investigation how we
>> can incorporate this stuff into the existing code.
 
> What type of project?
 
Embedded Linux controller.
 
> more powerful. This enhancement will cast its increased performance on
> other parts of the system, in the form of more load. Might end up making
> a set of locks contend more than they ever had before...
 
We are taking small steps and measuring as we go!
 
--
Ian
Jorgen Grahn <grahn+nntp@snipabacken.se>: Apr 03 12:27PM

On Sat, 2017-04-01, Chris M. Thomasson wrote:
> [...]
 
> How many people in this group are actually interested in this type of
> ability provided by C++?
 
I'm honestly more interested in keeping concurrency a thing between
processes.
 
I used lock-free data structures in one project, but got the feeling
that noone on the project (me included) could spot the bugs, and that
such bugs would mostly show up in production.
 
/Jorgen
 
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
"Chris M. Thomasson" <invalid@invalid.invalid>: Apr 03 01:36PM -0700

On 4/3/2017 5:27 AM, Jorgen Grahn wrote:
>> ability provided by C++?
 
> I'm honestly more interested in keeping concurrency a thing between
> processes.
 
That can be a great way to use the hornets nest Jorgen, in the sense
that the noexcept variety are basically immune to random failures. We
rely on the atomic instructions provided by the underlying hardware, to
be atomic. Also, we can use some of them in interrupt/signal handlers.
This makes them ideal for inter-process communication. Thread crash will
destroy its process. However, a processes dying should be handled, and
almost, "expected to happen" when dealing with inter-process sync.
 
Mutexs can play major role here, and should not be overlooked. I have
used Pthreads robust mutexs wrt EOWNERDEAD, and windows WAIT_ABANDONED
states before. Very nice. Fwiw, these states mean the calling thread has
access, but the shared data-structure needs to be validated, and
possible corrected.
 
In certain compatible lock-free algorithms, this correction does not
need to happen, and failures are fine!
 
 
> I used lock-free data structures in one project, but got the feeling
> that noone on the project (me included) could spot the bugs, and that
> such bugs would mostly show up in production.
 
The insidious nature of these things might not ever crash, or even show
a bug. For the random nature of a race condition can be very bad!
 
https://groups.google.com/d/topic/comp.lang.c++/7u_rLgQe86k/discussion
(read all if interested... ;^)
 
It might crash the program 42 years from now! However, it might give
slightly wrong results a several of times a month, if it feels like
it... Wow.
 
Very scary.
 
Give Relacy at try:
 
http://www.1024cores.net/home/relacy-race-detector
 
:^)
"Chris M. Thomasson" <invalid@invalid.invalid>: Apr 03 01:49PM -0700

On 4/2/2017 10:10 PM, Ian Collins wrote:
>>> can incorporate this stuff into the existing code.
 
>> What type of project?
 
> Embedded Linux controller.
 
Great! Talking out of ignorance because I do not know your goals/needs
on said project... On this type of device, I would take strong interest
in the single-producer/consumer relationship. No atomic RMW's!. A lot of
audio applications use this.
 
 
>> other parts of the system, in the form of more load. Might end up making
>> a set of locks contend more than they ever had before...
 
> We are taking small steps and measuring as we go!
 
Sounds totally prudent. Wonderful that I am "not alone" on this group!
Thank you Ian.
 
:^)
"Chris M. Thomasson" <invalid@invalid.invalid>: Apr 03 01:49PM -0700

On 4/3/2017 1:49 PM, Chris M. Thomasson wrote:
> on said project... On this type of device, I would take strong interest
> in the single-producer/consumer relationship. No atomic RMW's!. A lot of
> audio applications use this.
 
BTW, we can multiplex them.
 
bitrex <bitrex@de.lete.earthlink.net>: Apr 03 03:44PM -0400

Is it possible to modify the following code such that class "Bar"
contains a const array of "Foo" with a template size, and a stack
holding *consts into that array? That is to say something like:
 
std::stack<Foo *const> a;
const std::array<Foo, M> b = init_from_f<M>();
 
#include <array>
#include <stack>
 
struct Foo {};
 
template <size_t M>
struct Bar
{
Bar()
{
for (const auto& ptr : get_ptrs())
{
a.push(ptr);
}
}
std::stack<Foo*> a;
std::array<Foo, M> b = init_from_f<M>();
 
template <size_t N>
std::array<Foo, N> init_from_f()
{
std::array<Foo, N> a;
for (size_t i = 0; i < N; ++i) { a[i] = Foo(); }
return a;
}
 
template <size_t N>
std::array<Foo*, N> get_ptrs()
{
std::array<Foo*, N> b;
for (size_t i = 0; i < N; ++i) { b[i] = b.data() + i; }
return b;
}
};
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 03 10:25AM +0100

On Sun, 2 Apr 2017 11:29:34 -0700 (PDT)
 
> This functionality is 100% necessary for this library, but could be
> useful in other places. I do have an have an has_input_pending()
> method that uses poll, so if it's not possible then no worries.
 
You cannot do proper asynchronous (non-blocking) i/o using the standard
streambuffers. basic_streambuf::underflow() returns a signed integer
representing either the character at the start of the "pending
sequence" (the front of the buffered characters available) following the
read operation, or traits_type::eof() (normally -1) if end-of-file or an
error has arisen. As far as I can tell, underflow() returning 0 means
that a character of value '\0' has been received and is at the front of
the buffer, which would conflict with the fact that after underflow()
gptr() does not in fact point to that (non-existent) '\0' character.
The call to uflow() which invokes underflow() (and which does advance
gptr()) will therefore cause the character in fact at the gptr()
position to be lost.
 
In short, the standard streams (and streambuffers) expect a read
operation to block until the read request has been satisfied unless
either end-of-file has been reached or an error has arisen.
 
You could fudge it by ensuring that underflow() will never in
fact return with nothing added to the buffer. You could do
this by extracting a single character at a time from the buffer,
looping on basic_streambuf::in_avail() and when the buffer is empty
testing the underlying file descriptor for readiness with poll() or
select() in your asynchronous event loop. One of the problems with
this when using linux (if that is your OS) is that select() and poll()
on linux are not POSIX compliant and can spuriously report a file
descriptor as ready, so this arrangement could still block when you do
not want it to or (if the file descriptor is set non-blocking) cause
EAGAIN or EWOULDBLOCK to put the streambuffer in an error state when
there is in fact no error.
 
Chris
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 03 10:43AM +0100

On Mon, 3 Apr 2017 10:25:48 +0100
> > method that uses poll, so if it's not possible then no worries.
 
> You cannot do proper asynchronous (non-blocking) i/o using the
> standard streambuffers.
 
I should add that you _can_ do it with coroutines in association with
whatever asynchronous event loop you are using. What your underflow()
would do on encountering EAGAIN or EWOULDBLOCK is to yield on the
coroutine instead of returning, and that coroutine is then resumed by
the event loop when select() or poll() indicates that the descriptor has
become ready for reading.
 
I have written a library in another language (a language which supports
coroutines and explicit continuations), that does exactly that.
 
I believe coroutines did not make the cut for C++17, although I also
believe that Microsoft's VS does in fact support its own version of
them. So if you are programming with windows you could try writing your
underflow() using them.
 
Chris
Ron <ron.rwsoft@gmail.com>: Apr 03 04:58AM -0700

Thanks for the replies. It's pretty much as I first expected. As flexible as the stream API is in c++, it's proving to have some short comings. Right now this library is for Linux systems, but eventually will be ported to Windows via mingw. I don't use Windows enough to justify Visual Studios.
 
The coroutines suggestion has given me an idea of a possible internal event dispatcher when a socket is ready to read (or write). I think I saw something like this in tcl. This might work to prevent client side lockups. If a server doesn't respond exactly as expected, the blocking IO tends to be a problem.
Jorgen Grahn <grahn+nntp@snipabacken.se>: Apr 03 12:20PM

On Mon, 2017-04-03, Chris Vine wrote:
...
> this when using linux (if that is your OS) is that select() and poll()
> on linux are not POSIX compliant and can spuriously report a file
> descriptor as ready,
 
IIRC, this was (last time I looked, almost a decade ago) limited to UDP
sockets. But you're still right: they don't feel like promising POSIX
compliance here, so we can't rely on it.
 
> not want it to or (if the file descriptor is set non-blocking) cause
> EAGAIN or EWOULDBLOCK to put the streambuffer in an error state when
> there is in fact no error.
 
/Jorgen
 
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 03 02:14PM +0100

On 3 Apr 2017 12:20:43 GMT
 
> IIRC, this was (last time I looked, almost a decade ago) limited to
> UDP sockets. But you're still right: they don't feel like promising
> POSIX compliance here, so we can't rely on it.
 
The man page for select, which is imported into the man page for poll(),
says:
 
"Under Linux, select() may report a socket file descriptor as "ready
for reading", while nevertheless a subsequent read blocks. This
could for example happen when data has arrived but upon
examination has wrong checksum and is discarded. There may be other
circumstances in which a file descriptor is spuriously reported as
ready. Thus it may be safer to use O_NONBLOCK on sockets that should
not block."
 
As you say, if they don't feel like promising POSIX behaviour I will
not rely on POSIX behaviour, and it's not really a hardship: if you
are doing asynchronous i/o you must cater for EAGAIN and EWOULDBLOCK so
it doesn't really matter if very occasionally select() gives you
one more EAGAIN than you were expecting.
Chris Vine <chris@cvine--nospam--.freeserve.co.uk>: Apr 03 05:00PM +0100

On Mon, 3 Apr 2017 04:58:27 -0700 (PDT)
> saw something like this in tcl. This might work to prevent client
> side lockups. If a server doesn't respond exactly as expected, the
> blocking IO tends to be a problem.
 
There are coroutine libraries out there, for C at least. One which
comes to mind is GNU Pth. But then you risk imposing considerable
restraints on users of your library - I have never used Pth but it
doesn't look, for example, as if Pth has any provision for combining it
with native OS threads. One consequence is that nothing in the program
can block except by specialist functions which suspend such as
pth_read(), or everything will block. So once you put pth_read() in
your std::streambuf::overflow() function, all your i/o ends up having to
be done by the coroutine library.
 
(My coroutine library does use native threads, up to a point, in the
sense that user code can invoke native worker threads which can block
and there can be as many event loops as wanted each running in their
own native thread with each operating its own intra-thread
asynchronicity - say, one native thread per CPU. Intra-thread
synchronization uses CSP-style "meetings" and synchronization between
native threads uses message passing.)
 
The most promising synthesis of these kinds of ideas which I have seen
is this one: https://github.com/wingo/fibers/wiki/Manual . But that
isn't going to do you any good C++ wise, as it is written in a
different language and runs on a VM partly designed for the purpose.
Juha Nieminen <nospam@thanks.invalid>: Apr 03 06:04AM

> inline $f make_multidimensional_view( ref_<const Collections>... c )
 
Perhaps use C++, for starters?
"Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com>: Apr 03 10:11AM +0200

On 03-Apr-17 8:04 AM, Juha Nieminen wrote:
> Alf P. Steinbach <alf.p.steinbach+usenet@gmail.com> wrote:
>> inline $f make_multidimensional_view( ref_<const Collections>... c )
 
> Perhaps use C++, for starters?
 
Oh sorry. It is C++ but with use of a library. I think the code example
can be sufficiently understood to illustrate the problem, without me
using much time on replacing the library stuff with plain direct C++.
 
Here just replace `$f` with `auto`, and the `ref_` is self-explanatory I
should think. :)
 
Cheers!,
 
- Alf
Juha Nieminen <nospam@thanks.invalid>: Apr 03 06:12AM

In comp.lang.c++ Jeff-Relf.Me wrote:
> Then I showed you my "kooky and oddly formatted" version of it:
 
Who are you talking to?
 
> #define isLeapYear(y) ( !( y%400 ) || y%100 && !( y%4 ) )
 
And you immediately show the danger of using macros. Just consider what
happens if you call it eg. like isLeapYear(a+b).
 
Even when you fix the macro, there's still the problem of calling it
like isLeapYear(someReallyExpensiveFunctionThatTakesAMinuteToReturn()).
Jeff-Relf.Me @.: Apr 03 12:42AM -0700

"Chris M. Thomasson" <invalid@invalid.invalid>: Apr 02 05:56PM -0700

On 4/2/2017 2:31 PM, Ramine wrote:
 
> To solve this problem with a condition variable , of course
> the signaling of the condition variable must be inside the
> critical section of the producers.
 
Not true. And why would you use a condvar in a real time system anyway?
Try to go with wait-free all the way. ;^)
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)