Skip to main content

Final Project Part 01 - Making and Installing Dependancies for clib

To preface making clib I will first explain what it is. clib is a package manger, with the one major difference being it contains it's own dependancies and cannot be used to install anything else. The default clib repository is on git here. If you notice by default it has no hash or checksum function. To do this you must import it through clib. To make clib I used these steps:
  1. mkdir test
  2. cd test
  3. git clone https://github.com/clibs/clib.git
  4. make
  5. clib-install sha1
clib is now succesfully made and I have the sha1 function available to test. Inside the sha1 development repository here I notice it already had an extensive test file for function. I presumed getting this to work would be much easier than it actually was. The first thing to note, is the sha1 folder that gets pulled into the deps (dependancies folder) is much different than the development version. The published version has 3 files: package.json, sha1.c, and sha1.h. The development version has 7: .travis.yml, Makefile, README.rst, package.json, sha1.c, sha1.h, and test.c. The Makefile for clib does not allow compilation of the test.c file in the sha1 folder in deps. It will tell you that main has been defined somewhere else (clib.c). This means you will have to yank the sha1 folder out of clib entirely to test it.

After pulling the sha1 folder and compiling test.c it will likely tell you CUnit/Basic.h is undefined. This test file requires intalling CUnit. The biggest problem with CUnit is is it hosted on sourceforge here which only allows local downloading and our wget command does not work here. I was able to dowload it by using the command below:
This should naturally be done in an empty directory, which to get to CUnit you must do the command cd cunit/branches/mingw64. To make and install is tricky, first, you have to replace the config.guess file in the repository with yours from usr/share/automake.1.15. This will eliminate the error when running make that says it does not know where to build. Then run the commands to install:
  • automake --add-missing
  • autoreconf (these steps fixed some dependancy errors)
  • ./configure --prefix=/usr/lib/ (This is VERY important)
  • make
  • sudo make install
When you try to recompile the test.c file it will state the error that it cannot read line 0 of libcunit.so. By default without defining the prefix CUnit will install in usr/local which we don't want because the linker cannot find the library. We want it in usr/lib and then we want to run this commad from /usr/lib
  • sudo cp ./lib/libcunit.so ./
  • sudo ldconfig (finds new lib)
After this you should now be able to run make on test.c with no errors.

Comments

Popular posts from this blog

Final Project Part 02 - Final Summary

In conclusion, the -O3 flag was the most important discovery with trying to optimize clib. It offered a significant speed up with no interference, and provided the chance to uniform a many times used function, strdup. Overall the function is built extremely well with very advanced logic. Attempting to alter said logic sprouted many errors and warnings and left only simple compiler optimizations such as loop unrolling which made small differences in speed up. Clib as a whole is a great idea, offering many compartmentalized features for the C programming language that programmers could definitely find useful when developing. I hope in the future I can get more involved in writing code for open source projects such as clib whether that be doing optimization work or building from the ground up. This project not only gave me an insight on how these open source projects work but also at real world coding and improvement opportunities. I can honestly say now that I have had some experience

Final Project Part 02 - Sha1 Function Enhancements

To try to squeeze out a bit more performance I attempted to some compiler optimizations. Unfortunately, due to the sheer complexity of the algorithm, I was unable to find other logic complexities to simplify. I tried some loop unrolling to make the compiler have to work a little less, some examples are here below: I made a graph to demonstrate the minute differences this makes in the test vectors below: At most a few millisecond difference is all that can be acquired, and this is only from the finalcount[] array as the digest array produces errors if not compiled in a loop along with other for loops in the code. To test this I simply altered the sha1.c code and ran the make file to see if the vectors passed or failed. As mentioned this is a compiler optimzation, in other words it is completed already, especially at the -O3 level where the benchmarking was done. I would not  recommend this change to be pushed upstream normally due to the insignificant time change

Final Project Part 01 - Final Summary

To end part 1 I will summarize what information I have gathered for part 2: I am optimizing clib, a package manager. After benchmarking, it is clear there is a significant time delay in some advanced calls of the SHA1 function, such as ones that call update many times. To optimize, I am going to add the -O3 flag and remove a loop condition (currently). Some other observations: This project is relatively small with no ./configure for other platforms. The Sha1 code is unique and does not conform to the simple sha1 algorithm such as on    Wikipedia . The documentation (i.e. README) is relatively vague at describing the dependancies. It suggests only syntax that implies installation and isn't clear at documenting development vs. published code.   I have learned alot getting to this point in part 1. Firstly, I learned that library files can only be linked by using sudo ldconfig and the files must be in usr/lib. Secondly, I learned how to alter an advanced Makefile's fla