| Lynn McGuire <lynnmcguire5@gmail.com>: May 26 03:27PM -0500 "STL: Amazing Speed Differences between std::vector and std::set (Observed with an UndoRedoAction)" https://www.codeproject.com/Tips/5303529/STL-Amazing-Speed-Differences-between-std-vector-a I have seen this myself. We used a std::map for a very large set (>10,000 members) just because it is much faster than std::vector. Lynn |
| scott@slp53.sl.home (Scott Lurndal): May 26 08:44PM >https://www.codeproject.com/Tips/5303529/STL-Amazing-Speed-Differences-between-std-vector-a >I have seen this myself. We used a std::map for a very large set >(>10,000 members) just because it is much faster than std::vector. Isn't that computer science 101? O(N) vs. [O(1) .. O(Log N)] for vector vs std::map (red-black tree). |
| Lynn McGuire <lynnmcguire5@gmail.com>: May 26 05:06PM -0500 On 5/26/2021 3:44 PM, Scott Lurndal wrote: >> (>10,000 members) just because it is much faster than std::vector. > Isn't that computer science 101? O(N) vs. [O(1) .. O(Log N)] for > vector vs std::map (red-black tree). Sorry, never took CS 101 at TAMU. My degree is in Mechanical Engineering. I did take CS 204 or 304, IBM 370 Assembly Language Programming, for grins. Lynn |
| "Alf P. Steinbach" <alf.p.steinbach@gmail.com>: May 26 06:33PM +0200 On 2021-05-25 03:46, Lynn McGuire wrote: > some slop > fseek (pOutputFile, 0, SEEK_SET); > outputFileBuffer.reserve (outputFileLength); [snip] In the above code `ftell` will fail in Windows if the file is 2GB or more, because in Windows, even in 64-bit Windows, the `ftell` return type `long` is just 32 bits. However, the C++ level iostreams can report the file size correctly: ---------------------------------------------------------------------------- #include <stdio.h> // fopen, fseek, ftell, fclose #include <stdlib.h> // EXIT_... #include <iostream> #include <fstream> #include <stdexcept> // runtime_error using namespace std; auto hopefully( const bool e ) -> bool { return e; } auto fail( const char* s ) -> bool { throw runtime_error( s ); } struct Is_zero {}; auto operator>>( int x, Is_zero ) -> bool { return x == 0; } const auto& filename = "large_file"; void c_level_check() { struct C_file { FILE* handle; ~C_file() { if( handle != 0 ) { fclose( handle ); } } }; auto const f = C_file{ fopen( ::filename, "rb" ) }; hopefully( !!f.handle ) or fail( "fopen failed" ); fseek( f.handle, 0, SEEK_END ) >> Is_zero() or fail( "fseek failed, probably rather biggus filus" ); const long pos = ftell( f.handle ); hopefully( pos >= 0 ) or fail( "ftell failed" ); cout << "`ftell` says the file is " << pos << " byte(s)." << endl; } void cpp_level_check() { auto f = ifstream( ::filename, ios::in | ios::binary ); f.seekg( 0, ios::end ); const ifstream::pos_type pos = f.tellg(); hopefully( pos != -1 ) or fail( "ifstream::tellg failed" ); cout << "`ifstream::tellg` says the file is " << pos << " bytes." << endl; } void cpp_main() { try { c_level_check(); } catch( const exception& x ) { cerr << "!" << x.what() << endl; cpp_level_check(); } } auto main() -> int { try { cpp_main(); return EXIT_SUCCESS; } catch( const exception& x ) { cerr << "!" << x.what() << endl; } return EXIT_FAILURE; } ------------------------------------------------------------------------------- When I tested this with `large_file` as a copy of the roughly 4GB "Bad.Boys.for.Life.2020.1080p.WEB-DL.DD5.1.H264-FGT.mkv", I got [c:\root\dev\explore\filesize] > b !ftell failed `ifstream::tellg` says the file is 4542682554 bytes. - Alf |
| Lynn McGuire <lynnmcguire5@gmail.com>: May 26 01:28PM -0500 On 5/26/2021 1:36 AM, Christian Gollwitzer wrote: > surprisingly little to do to make it work. I have no idea what goes > wrong when you link to F77, though. > Christian We have casts all over the place that are killing us now. One of my programmers is currently converting us from ASCII to UNICODE at the moment and having all kinds of problems due to the casts. This program originated in 1987 with Windows 2.0 and C coding. The Win16 to Win32 port was a freaking disaster and took three of us 18 months to complete. Of course, a portion of our software was Smalltalk which was converted to C++ in that port. The calculation engine still runs as a separate program so the F77 code does not matter. Lynn |
| Lynn McGuire <lynnmcguire5@gmail.com>: May 26 01:30PM -0500 On 5/25/2021 11:16 PM, Bonita Montero wrote: > Maybe it would be an idea to process your file in pieces ? I have thought about that. Not today. I am thinking about trying the large address space switch though. Lynn |
| Lynn McGuire <lynnmcguire5@gmail.com>: May 26 01:32PM -0500 On 5/26/2021 11:33 AM, Alf P. Steinbach wrote: > !ftell failed > `ifstream::tellg` says the file is 4542682554 bytes. > - Alf I have already replaced the fell code with _ftelli64. // get the size of the output file fseek (pOutputFile, 0, SEEK_END); __int64 outputFileLength = _ftelli64 (pOutputFile) + 42; // give it some slop int outputFileLengthInt = (int) outputFileLength; fseek (pOutputFile, 0, SEEK_SET); Thanks, Lynn |
| Bonita Montero <Bonita.Montero@gmail.com>: May 26 08:37PM +0200 >> Maybe it would be an idea to process your file in pieces ? > I have thought about that. Not today. > I am thinking about trying the large address space switch though. If you're accessing the file lienary consider file-mapping. File-mapping is slower for random accesses since the pages have to be mapped on demand, but with linear accesses pre- fetching of your drive and the operating-system take effect. |
| Lynn McGuire <lynnmcguire5@gmail.com>: May 26 02:09PM -0500 On 5/26/2021 1:37 PM, Bonita Montero wrote: > File-mapping is slower for random accesses since the pages > have to be mapped on demand, but with linear accesses pre- > fetching of your drive and the operating-system take effect. I store a compressed copy of the output file in our binary file so that when the user sends it to us so we get a copy of exactly what happened. It is not a crisis if it is not there. It is a crisis if the file processing / storage causes our program to crash. Thanks, Lynn |
| Bonita Montero <Bonita.Montero@gmail.com>: May 26 05:54PM +0200 > That's one of the reasons we use pthreads instead of C++ threads. pthreads are really poor. F.e. having the opportunity to pass arbitrary parameter lists which might hold resources as elegant as a shared_ptr<> is much more powerful. And if you use pthreads you could also use std::thread( xxx, params ... ).detach() instead. |
| Bonita Montero <Bonita.Montero@gmail.com>: May 26 06:02PM +0200 > I'll have to add this to my list of why the C++ threading library is crap. Because of that ting ascpect ? Have you ever noticed the RAII-flexi- bility of C++-locking ? Have syou ever noted how convenient it is to pass arbitrary parameter-lists to your thread as it it would be a directly called function; compare the work of defining a structure, to define it, allocate it, fill it, pass it to the thread and deallo- cate it at the end of the thread - that's all for free in C++ and the performance is the same ! > I would suggest if your code doesn't need to be portable to > either use posix threads on *nix or its Windows equivalent. That's a lot of work more than with C++-threads. |
| MrSpook_c7o6A5w1@9wp2.net: May 26 04:28PM On Wed, 26 May 2021 17:54:06 +0200 >> That's one of the reasons we use pthreads instead of C++ threads. >pthreads are really poor. F.e. having the opportunity to pass Really? Want to have a guess what the C++ threading library on Linux uses? |
| Bonita Montero <Bonita.Montero@gmail.com>: May 26 07:04PM +0200 >> pthreads are really poor. F.e. having the opportunity to pass > Really? Want to have a guess what the C++ threading library on Linux uses? We don't discuss physical threading but how the language presents threading; and C++11-threading is by far more convenient than pure pthreads. |
| Paavo Helde <myfirstname@osa.pri.ee>: May 26 09:41PM +0300 26.05.2021 18:52 Bonita Montero kirjutas: > No, if the thread doesn't terminate before destruction or you > detach it an exception is thrown and if that is done while un- > winding your application is terminate()d. Right, it seems boost has formally deprecated the earlier detach() behavior in newer releases in favor of terminate(). This is controlled by the BOOST_THREAD_VERSION macro. By default this appears still to be defined to 2, which means detach(), and version 3 only appeared in 2012. So I should have said: "std::thread is based on boost::thread which automatically detached in the destructor, at least at the time when std::thread was standardized." |
| You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment