Thursday, August 8, 2019

Digest for comp.lang.c++@googlegroups.com - 25 updates in 6 topics

Frederick Gotham <cauldwell.thomas@gmail.com>: Aug 08 07:10AM -0700

I'm working on a firmware project at the moment, it's running on embedded Linux for a 64-Bit ARM processor (aarch64).
 
There's the main Makefile, which uses "make -C" to invoke further Makefiles in nested subdirectories. Altogether there's four thousand Makefiles.
 
I want to able to print out the function call stack, and so I've got code that calls "backtrace" and "backtrace_symbols".
 
I want to recompile the entire firmware, passing "-funwind-tables" to the compiler, and passing "-rdynamic" to the linker.
 
If all of my four thousand Makefiles were written properly, all I would have to do is:
 
export CPPFLAGS=-funwind-tables
export LDEXTRAFLAGS=-rdynamic
 
Unfortunately, not all four thousand Makefiles are written properly. Some of them have a line like "CPPFLAGS=-Wall", i.e. they use = instead of +=, and so this gets rid of my own special arguments.
 
I'm thinking maybe I should replace the compiler binary file, and also the linker binary file, with a one-liner script like this:
 
g++ -funwind-tables -rdynamic $*
 
Any suggestions on how best to do this?
Ralf Fassel <ralfixx@gmx.de>: Aug 08 05:54PM +0200

* Frederick Gotham <cauldwell.thomas@gmail.com>
| I want to recompile the entire firmware, passing "-funwind-tables" to
| the compiler, and passing "-rdynamic" to the linker.
 
| If all of my four thousand Makefiles were written properly, all I
| would have to do is:
 
| export CPPFLAGS=-funwind-tables
| export LDEXTRAFLAGS=-rdynamic
 
| Unfortunately, not all four thousand Makefiles are written
| properly. Some of them have a line like "CPPFLAGS=-Wall", i.e. they
| use = instead of +=, and so this gets rid of my own special arguments.
 
| I'm thinking maybe I should replace the compiler binary file, and also
| the linker binary file, with a one-liner script like this:
 
| g++ -funwind-tables -rdynamic $*
 
| Any suggestions on how best to do this?
 
cd /top/of/source/tree
find . -name Makefile | xargs sed -i.bak -e 's/CPPFLAGS *=/CPPFLAGS +=/'
 
Probably anchor the matches against beginning of line or somesuch.
 
Untested, of course.
 
HTH
R'
Szyk Cech <szykcech@spoko.pl>: Aug 08 06:35PM +0200

On 08.08.2019 16:10, Frederick Gotham wrote:
 
> I'm working on a firmware project at the moment, it's running on embedded Linux for a 64-Bit ARM processor (aarch64).
 
> There's the main Makefile, which uses "make -C" to invoke further Makefiles in nested subdirectories. Altogether there's four thousand Makefiles.
 
If you don't produce one exe from those makefiles then you can make your
life simply and use one CMake file. Then you can use CMakeLists.txt:
[!!! NOT COMPLETE EXAMPLE NOR TESTED !!!]
 
cmake_minimum_required(VERSION 3.9.0)
 
project(MyProject)
set(CMAKE_CXX_STANDARD 14)
 
option(WARNINGS_NOT_ALLOWED "Warnings not allowed when compile." OFF)
option(BUILD_TESTS "Build tests." OFF)
 
list(APPEND HEADER_SUBDIRS Include Src)
list(APPEND SOURCE_SUBDIRS Src)
 
# List headers
foreach(DIR IN LISTS HEADER_SUBDIRS)
list(APPEND HEADER_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.h)
list(APPEND HEADER_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.hpp)
list(APPEND HEADER_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.hxx)
list(APPEND HEADER_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.h++)
endforeach()
file(GLOB_RECURSE HEADER_FILES FOLLOW_SYMLINKS ${HEADER_EXPANDED_DIRS})
 
# List sources
foreach(DIR IN LISTS SOURCE_SUBDIRS)
list(APPEND SOURCE_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.c)
list(APPEND SOURCE_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.cpp)
list(APPEND SOURCE_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.cxx)
list(APPEND SOURCE_EXPANDED_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/*.c++)
endforeach()
file(GLOB_RECURSE SOURCE_FILES FOLLOW_SYMLINKS ${SOURCE_EXPANDED_DIRS})
 
add_executable(${PROJECT_NAME} ${SOURCE_FILES} ${HEADER_FILES})
 
# Add include dir
foreach(INCLUDE_DIR IN LISTS INCLUDE_DIRS)
message("INCLUDE_DIR: ${INCLUDE_DIR}")
target_include_directories(${PROJECT_NAME} PRIVATE ${INCLUDE_DIR})
endforeach()
 
# g++ compiler flags
if (CMAKE_COMPILER_IS_GNUCXX)
if(WARNINGS_NOT_ALLOWED)
message("Warnings not allowed.")
target_compile_options(${PROJECT_NAME} PRIVATE -Wall -Werror
-Wformat=2
-Wuninitialized -Winit-self -Wswitch-enum -Wundef
-Wpointer-arith -Wdisabled-optimization -Wcast-align
-Wcast-qual)
else()
message("Warnings allowed.")
target_compile_options(${PROJECT_NAME} PRIVATE -Wformat=2 -Wall
-Wuninitialized -Winit-self -Wswitch-enum -Wundef
-Wpointer-arith -Wdisabled-optimization -Wcast-align
-Wcast-qual)
endif()
endif()
Jorgen Grahn <grahn+nntp@snipabacken.se>: Aug 08 07:02PM

On Thu, 2019-08-08, Frederick Gotham wrote:
 
> There's the main Makefile, which uses "make -C" to invoke further
> Makefiles in nested subdirectories. Altogether there's four thousand
> Makefiles.
 
Someone somewhere should be fired ...
 
(I'm assuming you're not just seeing the internals of cmake or a
similar tool auto-generating a gazillion makefiles.)
 
> code that calls "backtrace" and "backtrace_symbols".
 
> I want to recompile the entire firmware, passing "-funwind-tables"
> to the compiler, and passing "-rdynamic" to the linker.
...
> also the linker binary file, with a one-liner script like this:
 
> g++ -funwind-tables -rdynamic $*
 
> Any suggestions on how best to do this?
 
Am I missing something? Sounds like you have a good solution right
there! Put the script(s) in the $PATH somewhere and you should be
fine.
 
/Jorgen
 
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
peteolcott <Here@Home>: Aug 08 12:15PM -0500

"This sentence is unprovable" can be proven to be unprovable
on the basis that its satisfaction derives a contradiction.
 
Proof that Wittgenstein is correct about Gödel
https://www.researchgate.net/publication/333907915_Proof_that_Wittgenstein_is_correct_about_Godel
 
--
Copyright 2019 Pete Olcott All rights reserved
 
"Great spirits have always encountered violent
opposition from mediocre minds." Albert Einstein
David Brown <david.brown@hesbynett.no>: Aug 08 12:51PM +0200

On 08/08/2019 00:51, Keith Thompson wrote:
> portability, you might as well use a word move instruction in
> assembly or a simple assignment in C++. You *could* break it down
> into byte moves, but the resulting code would be slower.
 
The distinction here must be between what is allowed by the language
(combining the standards and any compiler-specific defined features) and
what is allowed by the hardware. Many processors support unaligned
accesses. For some, these give significant speed penalties or
limitations (such as being non-atomic), while others have more efficient
support.
 
But C (and C++) does not support unaligned accesses. Most ways of
creating pointers to unaligned objects are undefined behaviour even
before you try to use them for access.
 
So the correct way to code this is using either portable constructs
(such as memcpy) which are given as byte moves, or using some
implementation-specific feature. It is then up to the compiler to turn
this into optimal code, such as using an unaligned move instruction. It
is not necessarily slower - indeed, on gcc you will get a single word
move instruction for a variety of different byte move combinations.
 
The nearest standard C has to saying "I want this to be a single word
move regardless of alignment" is to use volatile pointers. Compilers
rarely document what would happen in this case, but it would be an
extremely odd compiler that did not issue the access regardless of
alignment, invalid pointer casts, etc.
 
 
> For some CPUs, aligned access gives you better performance, but
> unaligned access still works. For others, unaligned access just goes
> kablooie.
 
Yes, that's a fine way to describe it!
Bart <bc@freeuk.com>: Aug 08 01:48PM +0100

On 08/08/2019 09:39, David Brown wrote:
> On 07/08/2019 23:52, Bart wrote:
 
> I don't see much connection between struct sizes and what we have
> discussing, but I will try to answer anyway.
 
I was picking up on the suggestion to just let a compiler get on with
its job. I'm saying that for things like this it will still need to be
checked, and if necessary to tweak a struct to get a more satisfactory
result.
 
That can include deliberately using some misalignment although usually
you will try and avoid that.
 
> (so that arrays of the struct work).
 
> So you won't get a struct array that has a stride of 65 bytes unless the
> struct itself has size 65 bytes.
 
The size might be 65 or more than 65, neither ideal if you were hoping
for 64.
 
(If I'm generating C, then I'd use pack(1) since the source language
does not do automatic padding, and then a size of 65 is possible for any
mix of element types. Although that would usually be picked up in the
original language before I start generating C versions.)
 
> course, but I have sometimes found it useful.
 
> The best method of checking struct sizes IMHO is with C11 static assertions:
 
> _Static_assert(sizeof(struct X) == 65, "Struct X should be 65 bytes");
 
I would just use printf("size = %<whatever>\n",sizeof(struct X)) while
creating it. Just to ensure it is some sane value.
 
> to avoid that in gcc:
 
> 1. Enable optimisation. That will coalesce the byte-at-a-time accesses
> on targets that support unaligned accesses.
 
On that specific example (that would have been on gcc 4.x on RPi in
2012), it was optimised, and the ARM32 processor /was/ capable of
unaligned access, but gcc still generated byte-at-a-time code).
Jorgen Grahn <grahn+nntp@snipabacken.se>: Aug 08 12:51PM

On Wed, 2019-08-07, Bonita Montero wrote:
 
[David Brown, attribution missing as usual]
 
>> know the OS will trap unaligned accesses and emulate them in software?
 
> It's just a tiny task to support this by an OS and helps to run a lot
> of old code; so this is very likely.
 
I take that as a 'no'.
 
(That aspect got lost in David Brown's frustrated response, which is
the only reason I bring it up.)
 
/Jorgen
 
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
David Brown <david.brown@hesbynett.no>: Aug 08 03:53PM +0200

On 08/08/2019 14:48, Bart wrote:
> its job. I'm saying that for things like this it will still need to be
> checked, and if necessary to tweak a struct to get a more satisfactory
> result.
 
Struct size and padding follows strict rules - it is not something that
varies between compilers (for the same target ABI), by versions or by
options. If you know the rules, the results are always predictable.
(Of course they can still be hard to check manually if the struct is big
or the member types are complicated).
 
Occasionally you will want to tweak a struct for better results. For
example, if it is of size 60 bytes, you might want to add a 4-byte dummy
entry so that it is 64 bytes giving more efficient use in an array
(especially when you arrange for it to have the same alignment as a
cache line). But that sort of thing is not very common.
 
> That can include deliberately using some misalignment although usually
> you will try and avoid that.
 
I don't imagine there is much use for deliberate misalignment. But
sometimes additional alignment (typically to cache line size) can be
helpful.
 
>> struct itself has size 65 bytes.
 
> The size might be 65 or more than 65, neither ideal if you were hoping
> for 64.
 
True. But the size and padding follows set rules, which allows you to
re-arrange fields to minimise padding.
 
> does not do automatic padding, and then a size of 65 is possible for any
> mix of element types. Although that would usually be picked up in the
> original language before I start generating C versions.)
 
I think if you want to translate directly from your source language into
C concepts, you should make your source language match the C rules here.
 
 
>> _Static_assert(sizeof(struct X) == 65, "Struct X should be 65 bytes");
 
> I would just use printf("size = %<whatever>\n",sizeof(struct X)) while
> creating it. Just to ensure it is some sane value.
 
IMHO, it is a lot better to have a compile-time assertion that is always
checked whenever the code is compiled, than a run-time check that you
confirm manually when you remember to include it and when you remember
to look at it. Printing the size of the struct can be handy if the
static assertion fails and you can't immediately see why - the static
assertion will tell you the struct is not 65 bytes, but it won't tell
you what it actually is.
 
 
 
> On that specific example (that would have been on gcc 4.x on RPi in
> 2012), it was optimised, and the ARM32 processor /was/ capable of
> unaligned access, but gcc still generated byte-at-a-time code).
 
It's hard to guess details from that description. But it is fair to say
the "enable optimisation" has no guarantees - gcc can spot common
patterns and optimise them, but not all possible patterns. And results
can vary by compiler version and target, with improvements for each
generation. "gcc 4.x" could be nearly 15 years old, or just 5 years old.
scott@slp53.sl.home (Scott Lurndal): Aug 08 02:46PM

>> David Brown <david.brown@hesbynett.no> writes:
>>> On 07/08/2019 14:47, Bonita Montero wrote:
 
 
>But C (and C++) does not support unaligned accesses.
 
Just a nit, but the _standard_ doesn't support unaligned accesses.
 
Every C and C++ compiler I've used supports them (both via some
compiler specific 'packed' attribute and via simple casts).
 
And while the behavior is undefined per the standard, it's quite
well defined per architecture if not portable.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 04:47PM +0200

>> Think of unaligned straddling a cache line!
 
> It is not the efficiency that is the main problem here - it is the
> correctness of the code (or lack thereof).
 
If it works as expected depends on the platform.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 04:50PM +0200

> aliasing rules so you have undefined behaviour anyway (although of
> course some compilers allow it with a suitable compiler flag such as
> -fno-strict-aliasing).
 
Doesn't matter! My code is written with conditional-compilation for
certain platforms where this does work. And it might be extended for
other platforms which also work.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 04:51PM +0200

> are actually supported by the compiler. You have identified some
> /targets/ that support unaligned access, but the /compilers/ do not
> support it.
 
The compilers I conditionally select suport it.
And the list might be extended.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 04:55PM +0200


>> It's just a tiny task to support this by an OS and helps to run a lot
>> of old code; so this is very likely.
 
> I take that as a 'no'.
 
No, that's a "mostly". But it's stupid to rely on that because it's
extremely inefficient - an article on the Web I found reported a slow-
down of about 300x over manually assembling the data-type like I did
it in the code I have shown here on Solaris / SPARC.
It's just a fallback for old code.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 04:57PM +0200

> extremely inefficient - an article on the Web I found reported a slow-
> down of about 300x over manually assembling the data-type like I did
> it in the code I have shown here on Solaris / SPARC.
 
Sorry, this wasn't correct. The benchmark was a comparison of
emulated unaligned accesses through trapping with aligned accesses.
But the differrence was about 300x.
David Brown <david.brown@hesbynett.no>: Aug 08 05:05PM +0200

On 08/08/2019 16:50, Bonita Montero wrote:
 
> Doesn't matter! My code is written with conditional-compilation for
> certain platforms where this does work. And it might be extended for
> other platforms which also work.
 
No - it is written with conditional compilation for platforms where you
/think/ it works. Speculation, guesswork and "it seemed okay when I
tried it" is not how you should be writing code.
David Brown <david.brown@hesbynett.no>: Aug 08 05:10PM +0200

On 08/08/2019 16:51, Bonita Montero wrote:
>> support it.
 
> The compilers I conditionally select suport it.
> And the list might be extended.
 
Show me links to the documentation supporting this claim.
 
I don't mean post examples of it working - I mean /documentation/.
Written information, from the compiler writers, saying that this sort of
code is valid for their compiler.
 
Until you can do that, you are an example of the cowboy attitude to
programming that crashes cars, brings down planes, and innumerable time
wasted and frustration caused when programs crash yet again.
 
It's one thing to have bugs in code by mistake, or because the
programming task is particularly hard, or because you don't know there
is a problem.
 
It is inexcusable to write code that you know is wrong and risky, just
because you are too lazy and arrogant to write it safely.
David Brown <david.brown@hesbynett.no>: Aug 08 05:14PM +0200

On 08/08/2019 16:46, Scott Lurndal wrote:
>>>> On 07/08/2019 14:47, Bonita Montero wrote:
 
>> But C (and C++) does not support unaligned accesses.
 
> Just a nit, but the _standard_ doesn't support unaligned accesses.
 
Unless you are talking about a specific compiler, the standard /is/ the
language.
 
> Every C and C++ compiler I've used supports them (both via some
> compiler specific 'packed' attribute and via simple casts).
 
A compiler supports features beyond the standards if it documents that
it supports them. Other than that, you are in the realms of "it worked
when I tested it" - i.e., luck and a problem waiting to happen.
 
So if your compiler says it works okay, that's fine - for that compiler.
If it supports "packed" attributes, or "__unaligned" keywords, use them
- that's fine.
 
 
> And while the behavior is undefined per the standard, it's quite
> well defined per architecture if not portable.
 
Sure, unaligned access is often defined on the underlying architecture.
But unless you are programming in assembly, that doesn't matter.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 05:33PM +0200


> No - it is written with conditional compilation for platforms where you
> /think/ it works. Speculation, guesswork and "it seemed okay when I
> tried it" is not how you should be writing code.
 
There's no speculation. It works.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 05:35PM +0200

> Show me links to the documentation supporting this claim.
 
There needs not to be any documentation on that. The thing is simply
that the compiler usually won't see that an access is unaligned in mot
times so it can't be incompatible to this. So the whole thing is simply
defined by the architecture and not by the compiler.
Bonita Montero <Bonita.Montero@gmail.com>: Aug 08 05:38PM +0200

And even more: compilers for those CPUs that support unaligned access
usually have special #pragma pack() or unaligned-directives. They won't
have these if they don't allow unaligned accesses.
"Öö Tiib" <ootiib@hot.ee>: Aug 08 07:10AM -0700

On Wednesday, 7 August 2019 17:41:47 UTC+3, Scott Lurndal wrote:
> >> threads without AIO?
 
> >It is better to use either async or blocking I/O but not to mix.
 
> For the same file descriptor or for the program? why?
 
I meant for a single stream, regardless if it is file or something
else.
 
> Use whatever is necessary to provide the required functionality
> and performance.
 
Exactly.
With tiny files of size measured in kilobytes it is cheapest
and most robust to read whole file and then to process the data.
If that starts affect performance then the issue is keeping data
in too lot of too tiny files.
With bigger files when either reading or processing what was read is way
more expensive than other or both are inexpensive enough (for
what is expected) then linear, blocking read and processing
cycle is good option. On case when it is large file and both cost of
reading and processing data affect performance then it is better
to keep input asynchronous and so to do these two things in parallel.
 
> >ourselves when glibc did it)?
 
> What's wrong with using blocking I/O for some files and async I/O for
> others? (or even both on the same file, for that matter).
 
I know of no example where it was optimal for same file.
I have experienced that attempts to mix the two for same file have
resulted with something pointlessly fragile and complicated. If
there is point to read parts of file asynchronously from processing
then switching between asynchronous and blocking mode have cost and
are complication. There have to be switching since file descriptors
that I've seen are not meant to do both at same time. Pointlessly
complex logic is then easier to break with maintenance. So it is
better to use blocking reads and to process the parts where processing
is expensive in separate threads instead.
Ralf Goertz <me@myprovider.invalid>: Aug 08 12:21PM +0200

Am Wed, 7 Aug 2019 21:20:36 +0200
 
> Backslash is used for set difference and a few other things in
> mathematics. Quotient groups use forward slash in my experience as
> does integer division. So I don't see any logic here at all.
 
I have the same experience. Maybe apart from integer division. I haven't
seen the forward slash as indicator of *integer* division but then I am
usually more interested in the remainder than the quotient. :-)
 
> However, sometimes mathematical symbol conventions vary by language -
> I have no idea about conventions in German.
 
The conventions in German don't differ from those in English in this
respect AFAIK.
Ralf Goertz <me@myprovider.invalid>: Aug 08 12:16PM +0200

removed with Claws Mail
Ralf Goertz <me@myprovider.invalid>: Aug 08 12:17PM +0200

removed with Claws Mail
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)