soft and program: comp.programming.threads - 16 new messages in 3 topics

comp.programming.threads
http://groups.google.com/group/comp.programming.threads?hl=en

comp.programming.threads@googlegroups.com

Today's topics:

* Lockfree queue, one producer many consumers... - 7 messages, 2 authors
http://groups.google.com/group/comp.programming.threads/t/1d57ec498b5268e9?hl=en
* experimental unbounded dynamic work-stealing deque in Relacy... - 6 messages,
1 author
http://groups.google.com/group/comp.programming.threads/t/11b38a3d2ca2a9f0?hl=en
* C++0x memory ordering question... - 3 messages, 1 author
http://groups.google.com/group/comp.programming.threads/t/4b22158fa78e801e?hl=en

==============================================================================
TOPIC: Lockfree queue, one producer many consumers...
http://groups.google.com/group/comp.programming.threads/t/1d57ec498b5268e9?hl=en
==============================================================================

== 1 of 7 ==
Date: Fri, Jun 26 2009 10:13 pm
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h248ev$o66$1@news.ett.com.ua...
> "Amine" <aminer@colba.net> wrote in message
> news:5576cc6c-d033-452c-9206-7a15ad0aafc1@l34g2000vbi.googlegroups.com...
> On Jun 26, 8:37 pm, "Chris M. Thomasson" <n...@spam.invalid> wrote:
> [...]
>
>> Hello Chris,
>
>> > I am wondering why `tail', `head' and `fMask' are arrays? Unless I am
>> > missing something, you only explicitly reference the first item (e.g.,
>> > tail[0], head[0] and fMask[0])...
>
>> I was testing and trying to avoid false sharing
>> by aligning - all the variables - at 64 bits , like a fool.. :)
>
>> have you any advice Chris ?
>
> Humm... Well, I am not all that experienced in Pascal. I can find my way
> around, but I am not expert enough to know how Pascal actually lays out
> data-structures in memory. Anyway, AFAICT it seems like your always going
> to have a "ping-pong" between the producer and consumers because the
> producer needs to read both the head and tail. So every time a consumer
> mutates the tail, that will have an effect on the producer. What am I
> missing here?

However, I would still try and pad the distance between the head and tail by
at least 128-bytes. Also, You might want to add padding before the head
itself. This will help prevent false-sharing from unrelated pieces of code
and the queue itself. But, since this is Pascal, who knows how the memory is
going to be exactly laid out. Perhaps you can enlighten me on this aspect of
Pascal...

== 2 of 7 ==
Date: Sat, Jun 27 2009 11:03 am
From: "Chris M. Thomasson"

"Amine" <aminer@colba.net> wrote in message
news:0ce1f268-1274-4982-ab75-430ed51efba7@r34g2000vba.googlegroups.com...
>
> Hello,
>
> I am still thinking and working on an algorithm for
> many producer many consumers.
>
> But here is an algorithm that works for one producer many consumers.
> it runs on Win , Linux and Mac (x86), just compile
> with Delphi or Freepascal ( http://www.freepascal.org/ )
[...]

I have implemented your algorithm in Relacy, but before I simulate it, I
need to know if I translated it correctly:
_____________________________________________________________________________
#define POWER 2U
#define SIZE (1U << POWER)
#define MASK (~(0xFFFFFFFFU << POWER))

template<typename T>
class spmcq {
rl::var<T> m_tab[SIZE];
rl::atomic<unsigned> m_head;
rl::atomic<unsigned> m_tail;

unsigned get_length() {
unsigned head = m_head($).load();
unsigned tail = m_tail($).load();
if (tail < head) {
return UINT_MAX - head + 1 + tail;
} else {
return tail - head;
}
}

public:
spmcq() {
m_head($).store(0, rl::memory_order_relaxed);
m_tail($).store(0, rl::memory_order_relaxed);
for (unsigned i = 0; i < SIZE; ++i) {
m_tab[i] = T();
}
}

public:
bool push(T& state) {
if (get_length() >= SIZE) {
return false;
}
unsigned tail = m_tail($).load();
m_tab[tail & MASK]($) = state;
m_tail($).store(tail + 1);
return true;
}

bool pop(T& state) {
unsigned last_head = m_head($).load();
for (;;) {
if (m_tail($).load() == head) {
return false;
}
state = m_tab[last_head & MASK];
if (m_head($).compare_exchange_weak(last_head, last_head + 1)) {
if (! (int)state) {
return false;
} else {
return true;
}
}
}
return true;
}
};
_____________________________________________________________________________

One thing that concerns me is the check to see if the popped item is equal
to zero:

if integer(obj)=0
then result:=false
else result:=true;

Does that mean that I cannot store a value of zero in your queue?

Any thoughts?

== 3 of 7 ==
Date: Sat, Jun 27 2009 1:51 pm
From: Amine

On Jun 27, 2:03 pm, "Chris M. Thomasson" <n...@spam.invalid> wrote:
> "Amine" <ami...@colba.net> wrote in message
>
> news:0ce1f268-1274-4982-ab75-430ed51efba7@r34g2000vba.googlegroups.com...
>
> > Hello,
>
> > I am still thinking and working on an algorithm for
> > many producer many consumers.
>
> > But here is an algorithm that works for one producer many consumers.
> > it runs on Win , Linux and Mac (x86), just compile
> > with Delphi or Freepascal (http://www.freepascal.org/)
>
> [...]
>
> I have implemented your algorithm in Relacy, but before I simulate it, I
> need to know if I translated it correctly:

all seems correct, please go ahead and simulate it.

> _____________________________________________________________________________
> #define POWER 2U
> #define SIZE (1U << POWER)
> #define MASK (~(0xFFFFFFFFU << POWER))
>
> template<typename T>
> class spmcq {
> rl::var<T> m_tab[SIZE];
> rl::atomic<unsigned> m_head;
> rl::atomic<unsigned> m_tail;
>
> unsigned get_length() {
> unsigned head = m_head($).load();
> unsigned tail = m_tail($).load();
> if (tail < head) {
> return UINT_MAX - head + 1 + tail;
> } else {
> return tail - head;
> }
> }
>
> public:
> spmcq() {
> m_head($).store(0, rl::memory_order_relaxed);
> m_tail($).store(0, rl::memory_order_relaxed);
> for (unsigned i = 0; i < SIZE; ++i) {
> m_tab[i] = T();
> }
> }
>
> public:
> bool push(T& state) {
> if (get_length() >= SIZE) {
> return false;
> }
> unsigned tail = m_tail($).load();
> m_tab[tail & MASK]($) = state;
> m_tail($).store(tail + 1);
> return true;
> }
>
> bool pop(T& state) {
> unsigned last_head = m_head($).load();
> for (;;) {
> if (m_tail($).load() == head) {
> return false;
> }
> state = m_tab[last_head & MASK];
> if (m_head($).compare_exchange_weak(last_head, last_head + 1)) {
> if (! (int)state) {
> return false;
> } else {
> return true;
> }
> }
> }
> return true;
> }};
>
> _____________________________________________________________________________
>
> One thing that concerns me is the check to see if the popped item is equal
> to zero:
>
> if integer(obj)=0
> then result:=false
> else result:=true;
>
> Does that mean that I cannot store a value of zero in your queue?

Just delete it.

>
> Any thoughts?

Amine.

== 4 of 7 ==
Date: Sat, Jun 27 2009 2:04 pm
From: Amine

On Jun 27, 12:51 am, "Chris M. Thomasson" <n...@spam.invalid> wrote:
> "Amine" <ami...@colba.net> wrote in message
>
> news:5576cc6c-d033-452c-9206-7a15ad0aafc1@l34g2000vbi.googlegroups.com...
> On Jun 26, 8:37 pm, "Chris M. Thomasson" <n...@spam.invalid> wrote:
> [...]
>
> > Hello Chris,
> > > I am wondering why `tail', `head' and `fMask' are arrays? Unless I am
> > > missing something, you only explicitly reference the first item (e.g.,
> > > tail[0], head[0] and fMask[0])...
> > I was testing and trying to avoid false sharing
> > by aligning - all the variables - at 64 bits , like a fool.. :)
> > have you any advice Chris ?
>
> Humm... Well, I am not all that experienced in Pascal. I can find my way
> around, but I am not expert enough to know how Pascal actually lays out
> data-structures in memory. Anyway, AFAICT it seems like your always going to
> have a "ping-pong" between the producer and consumers because the producer
> needs to read both the head and tail. So every time a consumer mutates the
> tail, that will have an effect on the producer. What am I missing here?

correct.

Amine.

== 5 of 7 ==
Date: Sat, Jun 27 2009 4:59 pm
From: "Chris M. Thomasson"

"Amine" <aminer@colba.net> wrote in message
news:2df52eab-64cd-4d49-974f-842d5dbc3648@g1g2000yqh.googlegroups.com...
On Jun 27, 2:03 pm, "Chris M. Thomasson" <n...@spam.invalid> wrote:
> "Amine" <ami...@colba.net> wrote in message
>
> news:0ce1f268-1274-4982-ab75-430ed51efba7@r34g2000vba.googlegroups.com...
[...]
> > I have implemented your algorithm in Relacy, but before I simulate it, I
> > need to know if I translated it correctly:

> all seems correct, please go ahead and simulate it.

[...]

Here is the simulation code:

http://relacy.pastebin.com/f6916a1f5

Okay. Relacy likes your algorithm if the number of pushed items never
exceeds the `SIZE' of the queue. It also likes it if I only use a single
consumer. However, when I increase the number of pushed items `ITERS'
__and__ use more than one consumer its giving me a data-race error between
the store in line (54) and the load in line (66); here is the exact output I
am experiencing:

http://relacy.pastebin.com/m6f5db774

This means that the producer is storing to the same place a consumer is
loading from at the same time. Please examine the algorithm and try to
ensure that I did not make some retarded boneheaded mistake wrt translation
from Pascal to Relacy C++!

So far, it goes like:

1 consumer == works fine
N consumers w/ total items never exceeding SIZE == works fine
N consumer w/ total items exceeding SIZE == data-race

== 6 of 7 ==
Date: Sat, Jun 27 2009 7:07 pm
From: Amine

On Jun 27, 7:59 pm, "Chris M. Thomasson" <n...@spam.invalid> wrote:
> "Amine" <ami...@colba.net> wrote in message
>
> news:2df52eab-64cd-4d49-974f-842d5dbc3648@g1g2000yqh.googlegroups.com...
> On Jun 27, 2:03 pm, "Chris M. Thomasson" <n...@spam.invalid> wrote:
>
> > "Amine" <ami...@colba.net> wrote in message
>
> >news:0ce1f268-1274-4982-ab75-430ed51efba7@r34g2000vba.googlegroups.com...
> [...]
> > > I have implemented your algorithm in Relacy, but before I simulate it, I
> > > need to know if I translated it correctly:
> > all seems correct, please go ahead and simulate it.
>
> [...]
>
> Here is the simulation code:
>
> http://relacy.pastebin.com/f6916a1f5
>
> Okay. Relacy likes your algorithm if the number of pushed items never
> exceeds the `SIZE' of the queue.

How can pushed items exeeds the size if we are testing with

if getlength >= fsize
then
begin
result:=false;
exit;
end;

Amine

>It also likes it if I only use a single
> consumer. However, when I increase the number of pushed items `ITERS'
> __and__ use more than one consumer its giving me a data-race error between
> the store in line (54) and the load in line (66); here is the exact output I
> am experiencing:
>
> http://relacy.pastebin.com/m6f5db774
>
> This means that the producer is storing to the same place a consumer is
> loading from at the same time. Please examine the algorithm and try to
> ensure that I did not make some retarded boneheaded mistake wrt translation
> from Pascal to Relacy C++!
>
> So far, it goes like:
>
> 1 consumer == works fine
> N consumers w/ total items never exceeding SIZE == works fine
> N consumer w/ total items exceeding SIZE == data-race

== 7 of 7 ==
Date: Sat, Jun 27 2009 9:37 pm
From: "Chris M. Thomasson"

"Amine" <aminer@colba.net> wrote in message
news:053d099a-6c64-44d4-9ac5-c048a0cdd515@h28g2000yqd.googlegroups.com...
[...]
> > Here is the simulation code:
> >
> > http://relacy.pastebin.com/f6916a1f5
> >
> > Okay. Relacy likes your algorithm if the number of pushed items never
> > exceeds the `SIZE' of the queue.

> How can pushed items exeeds the size if we are testing with

> if getlength >= fsize
> then
> begin
> result:=false;
> exit;
> end;

Yikes! I need to restate that...

If the number of attempted pushes never exceeds the maximum size of the
queue, Relacy likes it.

Anyway, please examine the code and tell me if I make mistake in your
algorithm.

==============================================================================
TOPIC: experimental unbounded dynamic work-stealing deque in Relacy...
http://groups.google.com/group/comp.programming.threads/t/11b38a3d2ca2a9f0?hl=en
==============================================================================

== 1 of 6 ==
Date: Sat, Jun 27 2009 12:44 am
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h24804$nv6$1@news.ett.com.ua...
> Here is a full-blown version of the experimental algorithm I invented in
> Relacy:
>
>
> http://relacy.pastebin.com/f2e9297b6
>
>
> Things are looking good so far. I can't seem to get Relacy to barf up any
> errors in the random scheduler with 99999999 iterations of 7 threads. I am
> currently running the test using the `rl::fair_full_search_scheduler_type'
> scheduler. So far so good!
[...]
>
> Memory barriers aside for a moment, a push never needs locks, and a pop to
> a deque with a size equal to or greater than WSDEQUE_SPLIT needs no
> locking as well. A steal operation is non-blocking wrt other stealers
> because it uses a try_lock operation to attempt to gain access. The times
> that a stealer will block the local owner thread is few and far between.
[...]

Humm... Speaking of memory barriers, I have extremely relaxed the membars in
the posted example:

http://relacy.pastebin.com/f5f819070

I only use acquire barrier to load from the counter variables, and release
to store to them. There is no #StoreLoad barrier on the fast-path. Relacy is
not puking out any data-race errors. I think I may have found a way to get
around the nasty #StoreLoad barrier in inherent in the non-blocking deques I
have seen. I need to do some more investigating. If indeed I found a way to
get around the expensive membar, well, this algorithm should really
outperform the one in Clik. Right now, I can't see any need for a
store-to-load pattern that has to be enforced. This is going to get
interesting...

;^)

== 2 of 6 ==
Date: Sat, Jun 27 2009 1:46 am
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h24ijj$rho$1@news.ett.com.ua...
> "Chris M. Thomasson" <no@spam.invalid> wrote in message
> news:h24804$nv6$1@news.ett.com.ua...
>> Here is a full-blown version of the experimental algorithm I invented in
>> Relacy:
>>
>>
>> http://relacy.pastebin.com/f2e9297b6
>>
>>
>> Things are looking good so far. I can't seem to get Relacy to barf up any
>> errors in the random scheduler with 99999999 iterations of 7 threads. I
>> am currently running the test using the
>> `rl::fair_full_search_scheduler_type' scheduler. So far so good!
> [...]
>>
>> Memory barriers aside for a moment, a push never needs locks, and a pop
>> to a deque with a size equal to or greater than WSDEQUE_SPLIT needs no
>> locking as well. A steal operation is non-blocking wrt other stealers
>> because it uses a try_lock operation to attempt to gain access. The times
>> that a stealer will block the local owner thread is few and far between.
> [...]
>
> Humm... Speaking of memory barriers, I have extremely relaxed the membars
> in the posted example:
>
> http://relacy.pastebin.com/f5f819070
>
> I only use acquire barrier to load from the counter variables, and release
> to store to them. There is no #StoreLoad barrier on the fast-path. Relacy
> is not puking out any data-race errors. I think I may have found a way to
> get around the nasty #StoreLoad barrier in inherent in the non-blocking
> deques I have seen. I need to do some more investigating. If indeed I
> found a way to get around the expensive membar, well, this algorithm
> should really outperform the one in Clik. Right now, I can't see any need
> for a store-to-load pattern that has to be enforced. This is going to get
> interesting...

Ahhhh... Relacy is detecting an access to freed memory error on this version
in another test I am working on (not posted yet) while the version with
sequential consistency works okay.

Humm... I need to do some investigating. I have to admit, its kind of a pain
to read through the lengthy output Relacy generates!

I am hoping that this algorithm turns out to work! Keeping fingers crossed.

;^)

== 3 of 6 ==
Date: Sat, Jun 27 2009 2:59 am
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h24m9q$s99$1@news.ett.com.ua...
[...]
>> I only use acquire barrier to load from the counter variables, and
>> release to store to them. There is no #StoreLoad barrier on the
>> fast-path. Relacy is not puking out any data-race errors. I think I may
>> have found a way to get around the nasty #StoreLoad barrier in inherent
>> in the non-blocking deques I have seen. I need to do some more
>> investigating. If indeed I found a way to get around the expensive
>> membar, well, this algorithm should really outperform the one in Clik.
>> Right now, I can't see any need for a store-to-load pattern that has to
>> be enforced. This is going to get interesting...
>
> Ahhhh... Relacy is detecting an access to freed memory error on this
> version in another test I am working on (not posted yet) while the version
> with sequential consistency works okay.
>
> Humm... I need to do some investigating. I have to admit, its kind of a
> pain to read through the lengthy output Relacy generates!
>
> I am hoping that this algorithm turns out to work! Keeping fingers
> crossed.

Okay, so far this is the best I can do without pissing Relacy off real bad:

http://relacy.pastebin.com/f272dbf1a

I changes the test code itself to better simulate a "real" work-stealing
environment. I change the members in the node class to use `rl::var' instead
of `rl::atomic' because no two threads can ever load/store from/to them
concurrently. Also, I change the head pointer to `rl::var' because only one
thread ever touches it. I remove some superfluous/unnecessary code that did
not need to be in there (e.g., the double-checking logic). As for memory
barriers, well, I can get away with using them in only a few key places. I
created macros so that I can easily change them:

#define WSDEQUE_LOCAL_RCOUNT_LOAD_MEMBAR rl::memory_order_consume
#define WSDEQUE_LOCAL_LCOUNT_STORE_MEMBAR rl::memory_order_acq_rel
#define WSDEQUE_REMOTE_LCOUNT_LOAD_MEMBAR rl::memory_order_consume
#define WSDEQUE_REMOTE_RCOUNT_STORE_MEMBAR rl::memory_order_relaxed

The first two macros deal with the local functions (e.g., push() and
pop()'), the other two deal with the remote function (e.g., steal()). If I
weaken the `WSDEQUE_LOCAL_LCOUNT_STORE_MEMBAR' to say, a release barrier,
Relacy freaks out and reports an access to freed memory error. In other
words, Relacy says the algorithm does not work when I do that. However, if I
leave the barriers as is, everything works good.

Well, at least the algorithm itself seems to be working.

BTW, can you think of any ways to get rid of the fuc%ing
`memory_order_acq_rel' barrier!!!???

Ouch! ;^o

== 4 of 6 ==
Date: Sat, Jun 27 2009 3:02 am
From: "Chris M. Thomasson"

Okay, so far this is the best I can do without pissing Relacy off real bad:

http://relacy.pastebin.com/f272dbf1a

Well, at least the algorithm itself seems to be working.

BTW, can you think of any ways to get rid of the fuc%ing
`memory_order_acq_rel' barrier!!!???

Ouch! ;^o

== 5 of 6 ==
Date: Sat, Jun 27 2009 3:29 am
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h24qh7$t2q$1@news.ett.com.ua...
> "Chris M. Thomasson" <no@spam.invalid> wrote in message
> news:h24m9q$s99$1@news.ett.com.ua...
[...]
> As for memory
> barriers, well, I can get away with using them in only a few key places. I
> created macros so that I can easily change them:
>
>
> #define WSDEQUE_LOCAL_RCOUNT_LOAD_MEMBAR rl::memory_order_consume
> #define WSDEQUE_LOCAL_LCOUNT_STORE_MEMBAR rl::memory_order_acq_rel
> #define WSDEQUE_REMOTE_LCOUNT_LOAD_MEMBAR rl::memory_order_consume
> #define WSDEQUE_REMOTE_RCOUNT_STORE_MEMBAR rl::memory_order_relaxed
>
>
[...]
>
> BTW, can you think of any ways to get rid of the fuc%ing
> `memory_order_acq_rel' barrier!!!???

Well, I tried something fairly odd and used an acquire barrier for the
stores:

#define WSDEQUE_LOCAL_RCOUNT_LOAD_MEMBAR rl::memory_order_consume
#define WSDEQUE_LOCAL_LCOUNT_STORE_MEMBAR rl::memory_order_acquire
#define WSDEQUE_REMOTE_LCOUNT_LOAD_MEMBAR rl::memory_order_consume
#define WSDEQUE_REMOTE_RCOUNT_STORE_MEMBAR rl::memory_order_relaxed

I also removed a store to the tail that I accidentally forgot to add relaxed
memory order. Here is the code:

http://relacy.pastebin.com/f7d79cd0f

Here is the difference between the new code and the last one I posted:

http://relacy.pastebin.com/pastebin.php?diff=f7d79cd0f

Now, here is my question... Is using an acquire barrier for a store even
legal in C++0x? I mean, what would that look like on a SPARC? Humm... I am
guessing that it would be a `#LoadStore | #LoadLoad'. Humm... But on the
other hand, its dealing with a store, so it still might add a nasty
`#StoreLoad'. Need to think...

Anyway, Relacy seems to like the code with the acquire barrier on the stores
because the test is passing the random scheduler with 99999999 iterations of
4 threads.

Any thoughts!?

;^o

== 6 of 6 ==
Date: Sat, Jun 27 2009 3:46 am
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h24s88$tfm$1@news.ett.com.ua...
[...]
> Anyway, Relacy seems to like the code with the acquire barrier on the
> stores because the test is passing the random scheduler with 99999999
> iterations of 4 threads.
>
>
> Any thoughts!?

Heck, I can even get it to work with a `memory_order_consume' barrier!

#define WSDEQUE_LOCAL_RCOUNT_LOAD_MEMBAR rl::memory_order_consume
#define WSDEQUE_LOCAL_LCOUNT_STORE_MEMBAR rl::memory_order_consume
#define WSDEQUE_REMOTE_LCOUNT_LOAD_MEMBAR rl::memory_order_consume
#define WSDEQUE_REMOTE_RCOUNT_STORE_MEMBAR rl::memory_order_relaxed

Is this a bug in Relacy or something, or is this legit wrt C++0x rules?

==============================================================================
TOPIC: C++0x memory ordering question...
http://groups.google.com/group/comp.programming.threads/t/4b22158fa78e801e?hl=en
==============================================================================

== 1 of 3 ==
Date: Sat, Jun 27 2009 3:33 am
From: "Chris M. Thomasson"

What does the following code produce on a SPARC:

std::atomic<unsigned> shared;

shared.store(1, memory_order_acquire);

Is that even a legal operation?

== 2 of 3 ==
Date: Sat, Jun 27 2009 3:41 am
From: "Chris M. Thomasson"

"Chris M. Thomasson" <no@spam.invalid> wrote in message
news:h24sf1$the$1@news.ett.com.ua...
> What does the following code produce on a SPARC:
>
>
>
> std::atomic<unsigned> shared;
>
> shared.store(1, memory_order_acquire);
>
>
>
> Is that even a legal operation?

Also, what is `memory_order_acq_rel' on a SPARC? is it something like:

MEMBAR #LoadLoad | #LoadStore | #StoreStore

or is there a #StoreLoad in there? Can it get away without using an `MFENCE'
on x86? How about on PPC, I assume it could be a `LWSYNC'...

I am asking this because I am trying to avoid a `#StoreLoad' membar in my
experimental work-stealing deque algorithm:

http://groups.google.com/group/comp.programming.threads/browse_frm/thread/11b38a3d2ca2a9f0

I can get the algorithm to work fine with a `memory_order_acq_rel', but am
worried that it will emit a performance destroying `#StoreLoad'! Ouch...

I can also get the algorithm to work with using a `memory_order_acquire' on
some critical stores, but I am still worried that its either no legal or it
will still emit a damn `#StoreLoad'.

== 3 of 3 ==
Date: Sat, Jun 27 2009 5:45 am
From: "Chris M. Thomasson"

Humm... I do not think that this is legal. Also, I think I read somewhere
that the following is also illegal:

shared.store(1, memory_order_acq_rel);

Why is that?

==============================================================================

You received this message because you are subscribed to the Google Groups "comp.programming.threads"
group.

To post to this group, visit http://groups.google.com/group/comp.programming.threads?hl=en

To unsubscribe from this group, send email to comp.programming.threads+unsubscribe@googlegroups.com

To change the way you get mail from this group, visit:
http://groups.google.com/group/comp.programming.threads/subscribe?hl=en

To report abuse, send email explaining the problem to abuse@googlegroups.com

==============================================================================
Google Groups: http://groups.google.com/?hl=en

soft and program

Sunday, June 28, 2009

comp.programming.threads - 16 new messages in 3 topics - digest

No comments:

Blog Archive

About Me