Thursday, October 20, 2016

Digest for comp.lang.c++@googlegroups.com - 6 updates in 3 topics

Vir Campestris <vir.campestris@invalid.invalid>: Oct 20 09:49PM +0100

> const char* hello;
> memcpy(hello, form, sizeof(form));
 
Those two lines, in English are:
 
1) Here is a pointer to some memory. It's not set up to point to any
particular memory, it's just to a random location in the system.
 
2) Copy the contents of "form" into the random location.
 
FWIW I always feel that in C++ if you're using a raw pointer you're
probably doing it wrong. Not always, but usually. And read about JSON
like Öö says!
 
Andy
Shivu <rout.jyoti@gmail.com>: Oct 20 02:04PM -0700

On Thursday, October 20, 2016 at 10:49:23 PM UTC+2, Vir Campestris wrote:
> probably doing it wrong. Not always, but usually. And read about JSON
> like Öö says!
 
> Andy
 
Thanks Andy.
Barry Schwarz <schwarzb@dqel.com>: Oct 20 03:44PM -0700

On Thu, 20 Oct 2016 21:49:11 +0100, Vir Campestris
 
>Those two lines, in English are:
 
>1) Here is a pointer to some memory. It's not set up to point to any
>particular memory, it's just to a random location in the system.
 
And perhaps more importantly, any data it eventually points to should
be treated as constant.
 
>2) Copy the contents of "form" into the random location.
 
And even when hello points to valid memory, attempting to update this
memory via a pointer to const causes undefined behavior.
 
 
--
Remove del for email
David Brown <david.brown@hesbynett.no>: Oct 21 12:18AM +0200

On 20/10/16 17:52, Jerry Stuckle wrote:
 
>> So to be clear on this, you understand that we are talking about
>> gigabits here, not gigabytes?
 
> For the pedantic arsehole, yes.
 
It is only a factor of 8 - hardly relevant, I suppose.
 
> because unless you have only one thing active in the entire system at a
> time, at some point you are going to have contention for data, slowing
> things down.
 
That's wrong for a start. You can only have one voltage /driving/ the
wire at a specific time, or only one voltage at any specific point on
the wire at any specific time. But as you note yourself, there can be
lots of bits in the pipeline along the wire. Also note that each symbol
in 10 Gb Ethernet (and therefore also 40 Gb, as it is basically 4 lanes
of 10 Gb in parallel) carries 4 bits in PAM-16 encoding. So the one
voltage point covers 4 bits. This also makes all your comments on
timing below out by a factor of 4.
 
But with those corrections, we can continue.
 
> inch - which means on a 100' cable, about 4800 bits would be transmitted
> before the first one was received at the destination. This slows the
> ability of the receiving system to respond to the transmitting system.
 
This all affects latency, but not throughput for large transfers.
 
You helpfully mention that the gross data rate of 10 Gb Ethernet is
actually 10.3125 Gb per second (a switching speed of 2.578125 GHz, with
4 bits per symbol). The reason for this is to cover the overhead from
Ethernet framing and spacing, so that the actual data transfer rate of
the Ethernet frame payloads is at 10 Gb/s for standard frame sizes.
 
> Then you also have to include any routers, switches, converters, etc. in
> the path, which cause additional propagation delays.
 
No, you don't have to include any of these in the path - because they
are not relevant to determining if a system can saturate its links. You
can do that using computer-to-computer links, without routers or
switches. If some part of your network cannot sustain full speed, then
of course it will limit the transfer speed - but that is not under
discussion here.
 
And if you have bought reasonable quality routers, switches and media
converters for your 10 Gb network, then they affect latency - but not
throughput for large transfers.
 
> the material being used in the semiconductors to voltage being used and
> stray capacitance in and around the circuit. They add delays to the
> signal, which are more significant as you go higher in speed.
 
Again, you are mixing up latency and throughput.
 
> I could continue, but we'll start with these. When you can figure out
> how to violate the laws of physics, you will win a Nobel Prize.
 
You can try again, if you like, and see if you can think of something
that affects throughput, and that is actually relevant to the discussion.
 
>> understand Ethernet with its framing overhead, inter-frame gaps, etc.
 
> You obviously do NOT understand about the physics involved well enough.
> There is much more than just TCP/IP's framing overhead, etc.
 
Actually, it seems to me that I understand this all a good deal better
than you do.
 
> And BTW - ethernet is a physical layer protocol. TCP/IP is only one of
> the link layer protocols which can be used.
 
I am aware of that.
 
> But even other protocols cannot not maintain 40Gb data transfer rates.
 
Using TCP/IP will mean you get a little under 40 Gb transfer rates,
because of the overhead in the protocol. But the overhead is nowhere
near the 25% you have been claiming. And TCP/IP demands more processing
power (for its extra CRC's) than simpler protocols like UDP or AoE - but
not more than Xeon's or the ThunderX cores can handle at this rate.
(Also, I would expect that the network cards handle the TCP/IP framing
and checking in hardware to offload the cpu.)
 
> itself. Once again, the laws of physics don't allow two resources to
> access the same memory chip/SATA device/whatever at the same time. One
> must wait.
 
You do need to find the data to transfer, that is certainly true. No
mention has been made as to where this data actually comes from. If Ian
or Scott are using these 10 Gb or 40 Gb links to test other network
equipment, then they can probably use generated or repeated test data
from memory - which has no problem keeping up with these link speeds (by
a factor of perhaps 500 times or more, depending on the number of DDR
channels). It has already been explained to you how disks systems can
give more than enough bandwidth to keep up.
 
>>> and 99% are once again theoretical limits - not real life ones.
 
>> Again, please explain what you think is missing here.
 
> I have.
 
I must admit I was pleasantly surprised to see that you tried. You
failed, as we all knew you would, but you get points for the effort.
 
 
>>> Do you like to your potential customers like this, also?
 
>> I can't even figure out what you are trying to say.
 
> Yes, you have already proved you don't understand the technical aspects.
 
By "technical aspects", do you mean basic English grammar such as "Do
you like to your potential customers" ?
 
Or do you mean the technical aspects of Ethernet that I understand and
you do not?
 
David Brown <david.brown@hesbynett.no>: Oct 21 12:33AM +0200

On 20/10/16 18:01, Jerry Stuckle wrote:
 
> You have just shown you don't have a clue about how it works. And no,
> the principles are *not* "all much the same". There are wide variations
> in how they work, depending on the requirements.
 
No, the principles /are/ the same. You have multiple disks in parallel.
You may spread the data over more disks to improve total bandwidth.
You may replicate the data over more disks to improve read bandwidth
(but not write bandwidth) and give redundancy. You may generate
parities for data integrity and recovery. You may combine disks to
increase the total storage.
 
Everything else about the different RAID types is different balances
between costs, space, bandwidth, redundancy, hardware requirements,
software requirements, rebuild times, etc.
 
 
> Yes, the laws of physics are much older than I am - as old as the
> universe, in fact. And they prevent two resources from accessing the
> same memory at the same time. One must wait.
 
And thus we have caches or buffers in order to change back and forth
between several slow paths and fewer faster paths while maintaining
throughput.
 
> of the throughputs of the individual disks. In reality, it is somewhat
> less than that, just like throughput on a single disk in reality is
> somewhat less than what is published.
 
/Somewhat/ less, I agree with - it can never give more throughput, so
any variations inevitably reduce the total throughput. But for
transfers that are big enough to make latencies a minor issue, you get
pretty close.
 
And as for the comparison between "real" speeds and "published" speeds -
that depends on the quality of the published data, the care taken by the
"real" user, and the matching of the workload between the real usage and
the testing usage.
 
> is based on the response of the slowest disk in the array. The
> "slowest" disk is based on the individual request, and can change with
> each request.
 
As elsewhere, you misunderstand the difference between latency and
bandwidth or throughput.
 
 
>> I'd ask for references or quotations to justify that, but we all know
>> how that will go.
 
> You don't remember what you claimed? No surprise there.
 
Since I did not make such a claim, why would anyone be surprised that I
can't remember it?
 
For numbers, the top PCIe SSDs read speeds of up to 3 GB/s (that's GB,
not Gb). Two of these (RAID0) will give you 6 GB/s, while a 40 Gb
Ethernet link is 5 GB/s. That is still very far from DRAM memory speeds
- I have never suggested anything else.
 
(Note - I have not tested such cards, so the numbers are from web
published benchmarks.)
 
 
>> I was explaining how you were wrong about port multiplexers.
 
> And you failed there, also.
 
Well, trying to teach you anything usually fails.
Marcel Mueller <news.5.maazl@spamgourmet.org>: Oct 20 09:43PM +0200

On 20.10.16 19.39, Mr Flibble wrote:
 
> The performance of map and multimap can often be improved significantly
> by using a pool allocator such as the one Boost provides. You should
> try using that rather than eschewing such standard workhorse containers.
 
You just mentioned that the standard allocator is often also not that
efficient when dealing with many objects. Even if the algorithms and
containers used for data processing and storage are suitable the overall
application performance may scale unexpectedly bad when the number of
allocated objects is large. So using a more efficient allocator for
std::(multi)map is just a work around for a not well chosen container
type and does not really solve the problem at all. Furthermore nested
allocators may significantly increase memory fragmentation in case the
container is held for a longer time with less than its maximum size.
 
The fact that there is no B-Tree in the standard makes no difference. RB
trees are only a good choice if invalidation of iterators on container
changes are undesirable. In most other cases they are outperformed by a
B-Tree by far. Think also of cache efficiency when small objects are
distributed over the entire memory.
 
 
Marcel
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.

No comments: