Monday, May 20, 2019

Serial sniffer and player with byte timestamp

Hello guys,

Recently I needed to sniff some serial data from an interactive session between a host and a serial device. I needed to playback this data at the same byte rate as when I recorded so I needed to record the data stream with timestamps and then play each byte at the correct moment in time based on the recorded timestamps.

This tool set is different than for example Grabserial because it creates a virtual serial connected to the sniffer, so the host program can still interact with the target through serial and have the stream logged. I hope I did not re-invent the wheel, but if I did let me know of a similar set of tools.

Also do not be harsh on the code. I did it as a tool for another project not a project itself. If you have improvement tips please let me know.
Also it is in my backlog to implement this tool so that it outputs a pcap file.

Sunday, January 27, 2019

Exporting an Amazon EC2 Instance and running it as a VM in QEMU

Some of you may know that exporting a live instance off EC2 is not easy. From all the limitations the most important is that the only way to export an instance is if it was imported from another virtualization environment in the first place.

In my opinion these area artificial limitations designed to make it harder for you to leave the Amazon eco-system. Legitimate, but not nice. It gets even worse due to the fact that the formats accepted are all connected with paid virtualization solutions.

As I do not care about paid virtualization solutions and just wanted to dump the instance as a way to keep a perfect copy of the instance for backup and make tests I decided to research on how to dump an instance the "monkey" way.

Dumping and running the instance locally

For my goals I consider an instance as a virtual private server with a main storage device. The storage device should contain all that is required to boot and start "as if" it was still running in the cloud.

There is a quick and dirty way, and there is a proper way

The quick and dirty involves making the disk dump while it is being used.  This will almost certainly lead to a slightly corrupted image, but it is more than recoverable if you call sync and are not doing any particularly intensive or important IO operation.

The proper way is just a few extra steps before the dirty way.
  1. Start a random instance to which you have ssh credentials and root access.
  2. Stop the instance which you want to backup.
  3. Detach the EBS volume from your main instance and attach it the newly created temporary instance.
  4. Start the temporary instance but do not mount the EBS storage.
  5. Get the device name of the EBS volume.
    lsblk
  6. Start of the quick and dirty way

  7.  On your local machine download the device contents.
    ssh -T $INSTANCE_ADDRESS 'dd bs=16M if=$EBS_DEVICE | bzip2 -c' | bzcat > $IMAGE_DESTINATION
    
  8. (Optional) Convert image to qcow2
  9. qemu-img convert -f raw -O qcow2 $IMAGE_DESTINATION $IMAGE_DESTINATION.qcow2
    
  10. Boot image on qemu.
    qemu-system-x86_64 -drive format=raw,file=$IMAGE_DESTINATION -enable-kvm -serial mon:stdio -vga virtio -device rtl8139,netdev=net0 -netdev user,id=net0,hostfwd=tcp::10022-:22,hostfwd=tcp::10443-:443,hostfwd=tcp::1080-:80 -m 2G
    

Notes and small explanations

Different compression algorithms

Instead of bzip it is possible to use other compression algorithms as well as different levels of compression. An example would be a gzip -> zcat pipe pair, or xz  -> xzcat pair.

 

 Convert image

What you download should be a raw disk image. You may want the disk image to be a more clever underlying storage like for example qcow2. You can have other formats though, including vdl, to use on Virtualbox application. See more information about qemu-img convert here.

 

System interaction

Contrary to the Amazon EC2, you can actually have live view of the booting process and kernel printing. This is quite useful to see if something went wrong with the dump. For example if you went with the quick and dirty way your file systems may need to be checked or some systems fail due to on-line file system dumping. This is in most cases survivable but you will be glad to have this information.

The command line part that redirects the kernel console to the console of the qemu application is
 -serial mon:stdio

Connectivity


QEMU connectivity and command line is huge and highly configurable. It is assumed that anybody doing work with an instance will want to connect to the machine through SSH or expose server services running there. There are mutiple ways to do it with different degrees of difficulty. In the qemu command presented above you can find that the ports of the Guest(Instance) will be exposed on the localhost.

  • Port 22 (SSH) of the Guest  mapped to localhost:10022
  • Port 443 (SSL) mapped to 10443
  • Port 80 (HTTP) mapped to 1080
Note that mapping ports to different values may lead to, for example broken web services. This is due to the fact that there may be automatic redirections on the browser side that discard the port numbers and will point to the Guest VM ports, which will then not match. The reason I did not match the ports of the guest directly to the localhost(host) is because some lower ports like 443 and 80 require root privileges, so qemu would need to be root.

Another trick to really fool all the redirections in you host to point to your virtual machine is to add an entry to the /etc/hosts file mapping for example a domain name of your real instance to your loopback address. This has worked really well for me but be aware that you need to disable after you are done otherwise you will be very confused :D

Wednesday, June 13, 2018

qemu serial device

I had massive trouble getting additional serial devices recognized by my kernel because for some very stupid reason (including me possibly being even stupider): the serial devices are attached as ISA devices by default. 

Yes, when you add the "-serial stdio" or something similar the serial device is on an ISA bus instead of the ultra-modern PCI. I have not verified this for myself but read here. Also I have verified that if I add the serial character device as a PCI device it magically is detected by the kernel.

As I am basing my kernel in multi_v7_defconfig, I cannot even find CONFIG_ISA in the menuconfig kernel configuration tool. I did not venture into manually adding it to the .config, as when the menuconfig does not show me love, i learned to be a gentleman. Also do not forget to add the serial driver, usable by QEMU, if you are building your own kernel .config. The relevant fragment is:

CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_DEPRECATED_OPTIONS=y
# CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_EXAR=y
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
# CONFIG_SERIAL_OF_PLATFORM=y


To run 'serial for tcp' but with a pci-serial I found this explanation.
 -chardev socket,host=127.0.0.1,port=59715,id=hostserial      -device pci-serial,chardev=hostserial

Sunday, March 04, 2018

ENV{DEVTYPE} does not seem to match

For some reason even though udevadm reports DEVTYPE, no udev rule matches when I try to use ENV{DEVTYPE}. Used ID_TYPE instead. Example udevadm report


UDEV [16712.707352] add /devices/platform/soc/soc:usb3-0/12000000.dwc3/xhci-hcd.3.auto/usb3/3-1/3-1.1/3-1.1:1.0/host0/target0:0:0/0:0:0:0/block/sda/sda1 (block) ACTION=add DEVLINKS=/dev/disk/by-id/usb-Lexar_USB_Flash_Drive_0FBC0FD4BECE8A47B3FE-0:0-part1 /dev/disk/by-partuuid/8ec91e13-01 /dev/disk/by-path/platform-xhci-hcd.3.auto-usb-0:1.1:1.0-scsi-0:0:0:0-part1 /dev/disk/by-uuid/662e6fe9-a29c-4a12-bc51-4a13c58b57c4 DEVNAME=/dev/sda1 DEVPATH=/devices/platform/soc/soc:usb3-0/12000000.dwc3/xhci-hcd.3.auto/usb3/3-1/3-1.1/3-1.1:1.0/host0/target0:0:0/0:0:0:0/block/sda/sda1 DEVTYPE=partition ID_BUS=usb ID_FS_TYPE=ext4 ID_FS_USAGE=filesystem ID_FS_UUID=662e6fe9-a29c-4a12-bc51-4a13c58b57c4 ID_FS_UUID_ENC=662e6fe9-a29c-4a12-bc51-4a13c58b57c4 ID_FS_VERSION=1.0 ID_INSTANCE=0:0 ID_MODEL=USB_Flash_Drive ID_MODEL_ENC=USB\x20Flash\x20Drive\x20 ID_MODEL_ID=a209 ID_PART_ENTRY_DISK=8:0fgf ID_PART_ENTRY_FLAGS=0x80 ID_PART_ENTRY_NUMBER=1 ID_PART_ENTRY_OFFSET=2048 ID_PART_ENTRY_SCHEME=dos ID_PART_ENTRY_SIZE=62535680 ID_PART_ENTRY_TYPE=0x83 ID_PART_ENTRY_UUID=8ec91e13-01 ID_PART_TABLE_TYPE=dos ID_PART_TABLE_UUID=8ec91e13 ID_PATH=platform-xhci-hcd.3.auto-usb-0:1.1:1.0-scsi-0:0:0:0 ID_PATH_TAG=platform-xhci-hcd_3_auto-usb-0_1_1_1_0-scsi-0_0_0_0 ID_REVISION=8.07 ID_SERIAL=Lexar_USB_Flash_Drive_0FBC0FD4BECE8A47B3FE-0:0 ID_SERIAL_SHORT=0FBC0FD4BECE8A47B3FE ID_TYPE=disk ID_USB_DRIVER=usb-storage ID_USB_INTERFACES=:080650: ID_USB_INTERFACE_NUM=00 ID_VENDOR=Lexar ID_VENDOR_ENC=Lexar\x20\x20\x20 ID_VENDOR_ID=05dc MAJOR=8 MINOR=1 PARTN=1 SEQNUM=44772 SUBSYSTEM=block USEC_INITIALIZED=712706913

Sunday, February 25, 2018

wic in yocto

As of the time of writing the manual of wic in the the Yocto Mega manual has a massive omission. It only explains how to use wic individually and does not talk about the IMAGE_FSTYPES = "wic". Because of this if you ran wic create you will get strange errors like.

Please make sure wic-tools have %s-native in its DEPENDS, bake it with 'bitbake wic-tools'

This message will not appear if you set IMAGE_FSTYPES = "wic", because the IMAGE_DEPENDS_wic already contains the above dependencies.

Also with the IMAGE_DEPENDS_wic_append_${MACHINE} you can deploy boot packages which wic could required to create the images.

PS: I hate blogger interface.

Edit: While grepping my poky I found that the included documentation refers to exactly what I wrote above. Well may be some googler will find it useful.

Wednesday, February 21, 2018

Nice trick to diagnose gcc search paths

Recently I needed to use a compiler as part of an sdk and needed to point it to the correct sysroot. For some reason the --sysroot flag did not work correctly. To inspect what was going on in gcc 'mind' a colleague told me about a "trick":

gcc -o  ./g.c -E  -P  -v -dD

Wednesday, October 04, 2017

External Source kernel modules install does not fail as it should

Another small post to serve for future memory that when you are building Linux Kernel modules from external tree source, the kernel build system behaves differently. Yesterday it happened that I lost a huge amount of time trying to understand why the installation of a module was not exiting with error, when I clearly saw errors in the logs. The reason is that this is on purpose. In the build scripts for kernel installation there is this hidden gem: https://github.com/torvalds/linux/commit/f6a79af8f3701b5a0df431a76adee212616154dc Don't stop modules_install if we can't sign external modules. I have been warned.

Monday, August 28, 2017

ParseError not a BitBake file

This error occurred to me when I made a require or include with the path to the file inside " quotation marks. The error message of course just said that  

ERROR: ParseError in "conf/machine/include/tune-cortexa53.inc": not a BitBake file

Solution: Remove the quotation marks around the file path in any require or include directives.

Tuesday, July 18, 2017

A script that pause/plays on lock/unlock Spotify

A script that pause/plays on lock/unlock
#!/bin/bash

dbus-monitor --session "type='signal',interface='org.gnome.ScreenSaver',member='ActiveChanged'" | \
(
  while true; do
    read X
    echo $X
    if echo $X | grep "boolean true" &> /dev/null; then
        dbus-send --print-reply --dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause
    elif echo $X | grep "boolean false" &> /dev/null; then
        dbus-send --print-reply --dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause
    fi
  done
)

Sunday, May 07, 2017

readlink system call fails on other files besides symbolic links

I just write this post to remind myself that the readlink system call fails on non symbolic links. I was expecting that it would work on files and directories but it doesn't.

man 2 readlink says in one of the errno:

EINVAL The named file is not a symbolic link.

So the next step is to check if the path is a symbolic link or not. The first choice would be to use stat on the path, but stat will give you results from the dereferenced file, not the symbolic link. So using stat system call on a file will never tell you if the file is a symbolic link.

lstat()  is  identical  to  stat(), except that if pathname is a symbolic link, then it returns
       information about the link itself, not the file that it refers to.


There you have it for future reference:

std::string MountHandler::doReadlink(std::string const& path) {
    char buff[PATH_MAX];
    struct stat stat_buf = {};
    if (lstat(path.c_str(), &stat_buf) == -1)
        throw std::runtime_error(fmtString("Could not stat path %s. Errno: %s",
        path.c_str(), strerror(errno)));

    if (!S_ISLNK(stat_buf.st_mode))
        return path;

    ssize_t len = readlink(path.c_str(), buff, sizeof (buff) - 1);
    if (len == -1) {
        throw std::runtime_error(fmtString("Readlink failed on %s. errno %s",
            path.c_str(), strerror(errno)));
    }
    buff[len] = '\0';
    return std::string(buff);
}

Sunday, April 16, 2017

Breaking abstract class interface in C++

Today while working on some code I had the need to make a tiny-bit-small break on a contract from an abstract base class.

The case is that I had an abstract base class called Pin, that contracted 2 methods for subclasses. The problem is that the InputPin class while benefiting from other functionality provided by the class Pin, could not sensibly provide a setPinValue method. Imagine I am naive and will only inform this operation is impossible on runtime:

volatile bool pinA1 = false;

class Pin {
public:
 virtual void setPinValue(bool value) = 0;
 virtual bool getPinValue() const = 0;
 virtual ~Pin() {}
}

class InputPin : public Pin {
public:
 virtual void setPinValue(bool value) { throw std::runtime_error("Cannot set value on input pin"); }
 virtual bool getPinValue() const { return pinA1 };
}

class OutputPin : public Pin {
public:
 virtual void setPinValue(bool value) { pinA1 = true; }
 virtual bool getPinValue() const { return pinA1 };
} 

Then I set out to find if I could use the compiler for my help in such a case. As usual Stackoverflow had question and answer about it. The answered marked as right is the ideal answer: No you cannot delete Pure Virtual Methods, fix your design. A bit further down showed the real answer: You can achieve something similar but it is not advised:

class InputPin : public Pin {
public:
 virtual bool getPinValue() const { return pinA1 };
private: 
 virtual void setPinValue(bool value) { throw std::runtime_error("Cannot set value on input pin"); } 
}


Just encapsulate the breaking method in the private part of the class and no one will be able to use it. If the Older-You gets the idea of using it in some private part of your own implementation you still get the luxury of the exception

I like answers who say "You kind of can do it, but you shouldn't", because you get to learn the corner cases and tricks with which to shoot yourself in the foot some time later. I, like everyone else, like a corner trick to impress the girls. Of course some time later it bites me back, or just my Older-Me looks back with astonishment at such a pretentious idiot I was.

*EDIT* After thinking a bit more about it and talking with my friend about it, he came up with the idea that just hiding the method in a private part of the class is not enough to make it reachable. This is because if the class is called polymorphically the virtual dispatcher in run-time will still reach this hidden method.

The best way in the end is to make it private and do nothing.

TLDR; Redesign your architecture to not have broken interfaces.

Saturday, April 15, 2017

Firefox mouse-less addon workflow

Today I had some time to tweak my Firefox workspace and came up with an addon combo which might be useful for somebody. The combo's purpose is to enable internet navigation almost completely without mouse.

Normally I have many open tabs on Firefox, and when there were that many they become invisible and only reachable by some kind of scrolling. This sucked because I do not have the habit of being organized and grouping tabs by theme, although I should.

The scrolling problem is removed by accepting that the tab bar can be multi-row. There just happens to be a cool addon called Tabmix Plus which allows multi-row tabs. It brings lots of other functionality which I do not care. If anybody is interested I exported it's settings for further reference are available in this link. To import it just go to TabMix Plus's settings and import the file.

This enables me tab visibility and knowing where the Tab I wanted was relative to my current tab, I coudl use the Ctrl-PgDn/PgUp trick switch tabs. The thing is, I have normally 30 tabs or more. Going one-by-one until my desired tab took sweet time. Also along the way it loaded all the tabs that were just lazy loaded.

I knew that there was a shortcut to go directly to a tab number. The problem is that this shortcut only goes up to the 9th tab. I found a very basic, but functional plugin to allow me to go to any tab number, "GoTo Tab Number". The plugin is very basic and with Ctrl-Shift-O I am able to raise an ugly dialog to write the tab number. Even so yet another problem came up: How do I know the tab number in 30 tabs?

I found yet another slightly abandoned addon caled "TabNumber". This one, I felt I had to modify it as the position of the numbers overlaying the tab just looked out of place. It took me quite long to adjust and find what I was comfortable with. This was because my addon development workflow to reload the changes was quite shitty:
  • Make changes to the extracted .xpi.
  • Zip the plugin contents into a xpi.
  • Copy the plugin to the where Firefox expected it.
In the end it worked and I created a repository in github to store it. I tried to contact the author about it and if he wants he can publish my changes although I doubt he will. The changes are GPL because the license is GPL. To install it, as stated in the README:
  • git clone https://github.com/ptsneves/TagNumber
  • cd TagNumber
  • zip -x "TagNumbersPTSN.xpi" -r "TagNumbersPTSN.xpi" .
  • Install the .xpi manually on Firefox.
For the navigation the VimFX plugin makes the honors. Very clean and I have yet to have any annoyance with it. My favourite and most used functionality is the F key to get a key combination for each possible link. When you type the appropriate key combination, firefox follows the link.

Normal Ctrl-LeftArrow allows to return to the previous page, and O allows the address bar to get focus with all current content selected. This way you can just start typing and what was there gets deleted automatically if you do not care about it. The Address bar also works as a search bar which makes it basically the only text are you need.

libpurple for facebook chat

Nowadays I use Facebook's messenger for about everything, to the point that even in the phone It has mostly replaced the SMS.
With this in mind, I would like a client for Linux that would natively connect to the Facebook Messanger. The intent is that I am not distracted by other Facebook content while doing more serious stuff. Also the web interface of messenger is bloody heavy.

Some time ago I used the facebook client of Thunderbird but it has since stopped working. I tried following the bugzilla ticket with no success. I even offered myself to pay to the author of the patch mentioned in the bugzilla ticket but when I made the offer I stopped getting any replies. I would have done it myself but....javascript... is definitely not the language I master. I feel Javascript is script kiddie language so it makes the will to dive into the code even lower. I am full of shit of course.

Sad, I went around for some more time using the web client of messenger until I found that there is a library, which makes Pidgin messenger compatible with Facebook's messenger. There is even an (outdated) Debian repository which makes the library installation painless (instructions here).

I used the library in the repository for some time but unfortunately it is not updated very often and recently the library broke with the error:
 Null value for $.viewer.message_threads.sync_sequence_id #306

For some weeks I was "sad" again as I did not want or know how to integrate the library build from source in my Ubuntu machine. This weekend I finally got some more time and bit the bullet, going on to find a fix for the problem. I found the solution thanks to the fast response from dequis and now leave here the instructions so that I can also remember when the time comes to update it again.

The following instructions to build dequis libpurple facebook plugin:


  • sudo apt-get install autogen autoconf libtool
  • git clone https://github.com/dequis/purple-facebook.git
  • cd purple-facebook
  • sudo apt-get install libpurple-dev libjson-glib-dev mercurial
  • ./autogen.sh
  • make
  • mkdir -p ~/.purple/plugin
  • cp pidgin/libpurple/protocols/facebook/.libs/libfacebook.so ~/.purple/plugins
I know there is a possibility to generate the debian packages, and it is probably as easy as, the previous steps, but I do not want to maintain any debian packages. Because of this  the library is now in an obscure part of my HOME directory which pidgin picks up, hihi.

Ok, ok. While I was writing this post I found it stupid to not write how to generate the debian package and gave it a try. Say you love me in the comments if it was useful: 

  • git clone https://github.com/dequis/purple-facebook.git
  • cd purple-facebook 
  • debuild -b -uc -us
  • cd ../
  • sudo dpkg -i purple-facebook_0.0.0-1_amd64.deb

Now I have a confession to make. I don't like Pidgin because it is slightly ugly. I use thunderbird and enjoy it, so my secret is that I would like to use this plugin for thunderbird. I set out to search a plugin for Thunderbird which would enable compatibility with libpurple and found an old plugin called "Additional Protocols for Thunderbird".
I tried to install it but it fails the compatibility check of my current Thunderbird. I even tried to be too clever, and downloaded an extension to Disable Compatibility check, but I couldn't understand if the plugin was broken or if I did not know how to use it.

Enjoy


Friday, March 16, 2012

How to Setup and run Ritchieduino

Hi again. The reason of this post is to show how to successfully set up the a developing environment to use Ritchieduino library. I will also assume some form of Arduino board available, running in Windows, even though any other platform will have just some adjustments. If you use for instance the ICSP mode you are on you own regarding avrdude.

To set up the environment the following software packages are required:
  • Ritchieduino package needed files which are here.
  • The latest winAVR, this is important because it provides the tools and configuration of the AVR-GCC tool chain. Don't forget to allow winAVR setup to change the your PATH variable so that the AVR eclipse can find it automatically. Please note that there may be a bug in which the setup destroys your pre-existing path. If such thing happens please go to the directory in which winAVR is installed and look for a file named path1.log which contains your previous PATH variable so that you can fix it.
  • The latest Classical Eclipse IDE, to use as the development environment. Before downloading it is necessary to check whether the Java virtual machine is 32 or 64 bits. To do that just make sure  Java is installed typing java -version. If the command successfully runs and there is not any mention of 64 bit in the output than it is a standard 32 bit Java Virtual Machine and you should download the 32-bit version of Eclipse. If the command fails it is possible you don't have java installed and you must install it from here. It is advisable to have the 32 bit Virtual Machine or there may be problems with RXTX plugin for Eclipse that allows the Serial console functionality

The following steps should be taken in order to make sure everything is set up correctly. The steps which are probably optional will be detailed for sake of completeness. More experimented users will probably be able to skip some steps.

  1. (Optional) This first step is to install the Arduino Windows drivers for the first time, so most users will probably be able to step over this. It is also really simple. The Arduino board should be connected to the USB port and  the"Windows key+r" be pressed at the same time. On the Run dialog write "devmgmt.msc. On the device manager window try to find "Ports (COM & LPT)". Under that is "Arduino UNO (COMxx)", where xx denotes any assigned port number (normally COM7). Right clicking and selecting "Update Driver Software" should open a dialog. Select "Browse my computer for driver software", the appropriate location for the Ritchieduino package should be selected. The drivers are in "\drivers\.inf". Hopefully the driver is successfully installed and the next steps can be taken.
  2. (Optional) The installation of winAVR is pretty much straight forward just keep in mind the problem with the PATH variable mentioned above.
  3. (Optional) The most critical and time consuming task is to install and configure the Eclipse IDE with everything a programmer needs to be productive.The first thing to be done after downloading Eclipse IDE is to extract it to somewhere. After that, it open it for the first time.
  4. (Optional) Upon starting Eclipse for the first time a directory should be selected where the projects will stay. Eclipse calls this directory workspace. It is advisable to either leave the default directory set or create one specifically for this purpose, so that everything stays organized in one place without future conflicts, like with Git. The check box  "Use this as the default and do not ask again" should also be activated so that any accidental change doesn't break the set up. Upon proceeding you will be officially inside the Eclipse IDE and will be greeted by a "Welcome to Eclipse" screen. Congratulations!
  5. (Optional)  In this step necessary plugins will be installed in Eclipse, specifically the CDT, AVR, EGit, RXTX and Target Management Terminal.
    1. So on the menu bar select "Help->Install New Software". When the new dialog appears, click on "Add" which will open a dialog to add a repository. In this case, the repository to add is the AVR plugin. The name can be anything, but for the sake of completeness it will be named "Avr Eclipse Repository". The location will be "http://avr-eclipse.sourceforge.net/updatesite". Note that this address can change, although unlikely.
    2. On the combo list "Work with:" the "Avr Eclipse Repository" should be selected. It is likely that a "Pending" will appear on the list below. It can be a while, as Eclipse is fetching the available plugins. 
    3. Afterwards the "AVR Eclipse Plugin" should show on the list and its check box should be activated. 
    4. Pressing "Next" twice and followed by accepting the license should highlight "Finish" which should be pressed to start the plugin installation process. 
    5. You should wait while software installs. During the installation it is possible that Eclipse warns you of unsigned content and you should continue the process. If you need re-assurance follow to the original AVR plugin site and check that the address supplied before matches the recommended in the site.
    6. Upon request restart Eclipse IDE.
    7. The process of installing another plugin will be repeated now for CDT, the component that adds general C/C++ functionality to the IDE.
    8. Following the Help->Install New Software" receipt again, select on the combo list "Work with:" the "All Available Sites".
    9. Below the combo box should be an empty text box which acts as a filter. There the text "C/C++ Development Tools" should be pasted. Note that if you manually write it, Eclipse IDE may hang because it is filtering the whole plugin list aggressively character by character. Even so it might hang some time.
    10. Tick only the "C/C++ Development Tools" and proceed like in the AVR plugin installation (step 5.5). In the end of this step it would already be possible to target AVR hardware but regarding Ritchieduino some more steps are required.
    11. Again the process will be repeated, only now to install eGit which enables the cloning of the Ritchieduino repository in Github.
    12. To install EGit the instructions are similar to the ones in step 8 just that this time the filter should be "Eclipse EGit" proceeding with similar previous steps to install it.
    13. Similar to the procedure in Step 5.1 followed to add the AVR plugin, RXTX repository is required to be added. To do so add "http://rxtx.qbang.org/eclipse/" selecting afterwards RXTX 2.1-7r4 and installing it. 
    14. This time the procedure is similar to the installation of EGit, just write in the filter "Target Management Terminal" and proceed with the install.
    15. The plugins installations are complete and it is time to start cloning the repositories
  6. In this section the Ritchieduino clone will be done.
    1. It is possible to clone the Ritchieduino repository by following "File->Import->Git->Projects from Git->Next" and then a dialog will ask you to select the repository. For that click "Clone"  and a dialog with lots of text boxes will appear. 
    2. Fortunately only the URI one is needed and in it input "git://github.com/ptsneves/Ritchieduino.git" and all other fields will automatically be set. Clicking "Next" will connect to the repository and Master branch will be already pre set. Continue with "Next" and "Finish" the repository will be set up. The dialog will close.
    3. If the repository was successfully added, the Ritchieduino repository should now be selected proceeded with "Next".
    4. Check that the "Import existing projects" radio button is set and just press "Next", and "Finish" in the next page.
    5. If everything has gone well the Ritchieduino default Projects should have been imported, making one more step complete.
  7. In the following steps the AVR environment will be configured according to the required AVR target chip.
    1. Starting by going to to the menu bar "Window->Preferences".
    2. In the opened dialog collapse the "AVR" tree-list and select "Paths".
    3. If the winAVR installation was successful all the paths should be already pre-set, with the exception of the Atmel Part Description Files. In the list find "AVRDude" and click on "Edit".
    4. On the "Change Path for AVRDude" change the "Path Source" to "Custom". Afterwards select "Browse...". Selecting the Ritchieduino Software Package folder that was previously downloaded is all that is necessary. 
    5. Back in "Preferences", now looking again at the tree list, and under "AVR", select "AVRDude".
    6. Mark  "Use custom configuration file for AVRDude" and "Browse", selecting again the Ritchieduino Software Package folder.
    7. Now on the "Programmer configurations" press "Add..." .
    8. A big dialog should open, and in the "Configuration Name" Ritchieduino should be written.
    9. The "Description" can be anything.
    10. On the "Programmer Hardware" list, "Arduino" should be selected.
    11. The "Override default port (-P)" text box should be set with \\.\COM7. (Without the dot).As mentioned earlier it can be other COM number, but normally its 7.
    12. The "Override default baudrate (-b)" should be 115200.
    13. Proceed with "Ok" until the "Preferences" dialog closes.
  8. Now there is a quite boring task that has to be repeated for all the libraries that you are going to use in Ritchieduino, but in fact is crucial and can't be defaulted, which is setting the target chip. For the sake of simplicity only the steps for RitchieduinoCore.
    1. On the menu bar go to "Window->Show View->Project Explorer"
    2. On the Project Explorer left click on the RitchieduinoCore project and select "Properties".
    3.  On the tree list select "AVR->Target Hardware".
    4. On the "MCU Type" you should select your target chip and in the "MCU Clock Frequency" the frequency. For instance for the Arduino Uno, the type is "ATmega328p" and the frequency "16000000".
    5. This whole step should be repeated for every library you wish to use once and every new project you decide to create. If this step is not taken your applications may not work correctly.
  9. There is a project ready to try which is the RitchieServer, an example ready to run. It is important that step8 is done on this project also. Note that the Ethernet shield is necessary.
    1. Go to the menu bar and select "Window->Open Perspective->Other". On the dialog that appears select "C/C++".
    2. On the Project Explorers' tree list collapse and double click "RitchieServer->main.c" so as to see what is being executed.
    3. Care should be taken so that the network settings in the source file match the desired local network.
    4. To build this project you only need to go to the Project Explorer, right click RitchieServer and select "Build Project".
    5. It will take some time in the first run but should be relatively fast afterwards (faster than the Arduino IDE).
    6. Upon reading "**** Build Finished ****" on the console vie.
    7. If everything has been successful it is now possible to upload to the board.
    8. To upload the program just select the RitchieServer project in Project Explorer and click on a green AVR icon on the tool bar in the upper part of the Eclipse window. Alternatively it is possible to do "Ctlr+Alt+U"
    9. After the upload to see the results of the Serial port data doing "Window->Show View->Terminal" is needed.
    10. After the terminal shows in the tabs in the view sub window there is a small button called "Settings".
    11. In the dialog that pops up the "Connection Type:" should be set to "Serial" and the "Port" should be set to COM7 or if different to the relevant one. The baud rate should be noted in future uses. Pressing "Ok" closes the dialog.
    12. Pressing the small green "Connect Button" connects to the serial device.
  10. If everything has been successfully executed the terminal should now be printing, for the case of RitchieServer, "Comunications Initted".
That is all that is needed to set up Ritchieduino. There will be another post on how to make a Hello World be printed in the Serial output.



Wednesday, March 14, 2012

Ritchieduino, What it is and why



Hello, this is my first post about the library i have created exclusively in C99 for the Arduino board, even though it might work with other AVR chips also and outside the Arduino board. I had already made a post on Arduino Forums, where a brief explanation can be found.
The Ritchieduino comprises much of the Arduino-0022 library functionality (pin compatibility), just completely written in C99 (not K&R C) and with some re-arrangement to make the code more modular. Also all the C++ code was mostly stripped off and the more important parts were partially re-written, namely the Serial, the SPI and Ethernet (w5100) libraries. Because of this re-writing the API changed significantly and i admit it is aimed at a more savvy audience than the original Arduino.

On the other hand as the including headers have been streamlined and the namespace cleaned some room for extra optimization control was gained. A clearer example is that mostly only the wiring library remained, which was already C. There were also some functions which were, in my opinion superfluous, that were cleaned out, namely re-naming of standard C functions, like the ones in WCharacter.h and WMath.h. I do reckon renaming is mostly inlined and doesn't bloat the code by itself but makes the analysis and deeper understanding of what is going on behind scenes mores obscure.

Still on the line of modularilization is the fact that in my opinion, Class Inheritance has been abused as to make Matroska doll code, where you have to go deeper and deeper to inspect what is happening under the hood, becoming even harder for newbies to start progressing to Intermediate level. An example of this is the print inheritance on pretty much every class, which sometimes calls code that is less than desirable, namely floating point code, which increases the size of the programs by some orders of magnitude.

As a final personal note i would like to notice that in no way i am diminishing C++, on the contrary. I think it is a much more powerful language with some concepts that, if not well understood and disciplined, can lead to quite obfuscated code. That and the fact that there doesn't seem to be any discipline regarding coding style as simple as a variable naming convention. A good example is in the socket.c file, where the socket descriptor variable can be called s, _s, sock or socket. The convention followed by Ritchieduino and its libraries attempts to follow as closely the Google Coding Style Guide as possible, even though there are exceptions(not the C++ ones) in the name of compatibility with the original Arduino library. I also advise a look into the guide even if you are not interested in Ritchieduino because for me it was really useful. It is even for C++!

There will be more posts on how to start and use the libraries, as it doesn't use the Arduino IDE, which in my opinion is terrible. I will make a post on how to set everything up in Eclipse IDE with some hints that may help the transition to Ritchieduino and even normal Arduino programming a bit faster and satisfying. In the meantime if you can take care of using the library you can try the WebServer examples which is functionally equal to the one provided in the original Ethernet library.

In the meantime i leave here the github repository with library available to anyone who so desires. It is also important to note that the library is LGPL, because the previous Arduino work upon which Ritchieduino is derived, is also LGPL, so you can use it in proprietary applications.

Hope enjoy it.
Paulo Neves




Saturday, February 04, 2012

Technological improvement has been the best way to change the balances between rich and poor, big and small. In the beginning of last century, technological breakthrough often came from small companies that grew to behemoths today. That breakthroughs were mostly in the mechanical field and chemistry but they all started from small family businesses changed or created new markets.  One thing they had in common was the fact that the success often came from the brainchild project of some inventor.

Nowadays technological breakthrough comes from the fields of computer science. It is known that lots of small companies start up on this field now but they are almost always swallowed by bigger players, often state backed or field founders, like Microsoft or Apple. Even so it is possible to argue that the IT industry provides lots of opportunities for entrepreneurship.

Never in history had mankind access to so much knowledge in a wide range of areas, so it reasonable to ask why are there not any new big companies growing from small businesses, considering the unprecedented number of highly educated individuals active in our society

There are many talented people with brand new ideas that take advantage of completely new ways of solving problems. The problem of our time is just that entrepreneurship is borderline impossible in terms of necessary investment and competition. Initial investment is a problem because most people who create disruptive technologies are young, making them unlikely to have the economic assets to even take care of the legal part of a business. Competition because bigger companies have an established business which takes considerable effort to make it that way.
 
Taking the late 19th century success stories of the Diesel engine and its self made man, and the DuPont company which started from nothing, not many new companies rose to stardom without coming from strong business conglomerates. Globalization did in fact create many new challenges to the developed world, specially regarding how to create companies that have to abide by stricter laws (labor, environmental, legal) and that the same time be competitive. On the other hand a developed society has a bigger human capital, as in creative, which it needs to tap effectively to keep survive.

Established companies still need to compete and at some level innovate at the lowest pace possible as not to be obsoleted. With this in mind the concept of idea contests took place, offloading the research and development to the contest participants. The contest takes place, the company funds the prize, and not surprisingly absorbs the idea(s) and the inventor into its structure, actually removing the risks associated with R&D . This idea may look great in the sense that both parties benefit from the contest, altough that may not be true.
As the natural law states, a body will remain in rest until some force acts upon it. With this in mind we consider a stable market where there is no immediate force acting asking for the company to react, so there is no reason for the company to take the risk of using a novel idea/product when nothing compels it to. In consequence, the company will obviously not take the unnecessary risk and will shelve the idea. This loss is invisible to the whole system as this new market changer invention, never came to be, but the market and ultimatelly the people will suffer for it.
On the other hand if we actually tip the system into favoring this new inventor into creating their own business of which they have everything to gain, then there would be a bigger chance of seeing all the unprecedented education investment, returning in the form of growth, employment and taxes. Perhaps the contests would not be needed if there was a visible network that inventors/enterpeneurs could easily get assistance from.



Wednesday, January 02, 2008

My experience with Lenvo Thinkpad X61t
Hi as this is my first English post in a Portuguese blog i will just say that i'm a Portuguese guy that just finished his high school and i am now attending classes so i can get the air line pilot license.
In Portugal Lenovo doesn't sell their computers in single units, only in large quantities that usually are associated with government and high profile corporations. But as i'm a stubborn geek alike guy i found out about the tablet world and since then i knew that a tablet was the way into the future and like a swiss knife for just about everything. I then started searching the Internet for the greatest value for my limited budget, and found that X61t was everything i needed. I then started looking for any European shop that sold IBM, but it seems that IBM/Lenovo dont really want their computers on the street in Europe, as such i had to wait a lot of time so i could have a pilot friend that regularly flies to the United Stated to arrange a way to order the tablet, but even in the United States Lenovo isn't exactly a flasher.

About the tablet:
I got the cheapest set but i'll be damned if it isn't worth every penny i spent on it. I go only on the negative aspects as it's the hardest thing to find.
First of all it comes with the integrated intel graphic card that really sucks if you want to play some modern strategy game with the pen enabled. Note that this is only a negative aspect if you consider you sometime want to play some modern game to relax.
Second, this convertible is a highly mobile machine slim, light and cute, and as such it probably would have won some points if it included in the basic model the enhanced battery. Note that this is just a problem of $$ because you can buy a separate battery unit that has got 8 hours of autonomy. Plus the original battery can hold the machine for little over 2 hours.
Third, consider that this laptop comes with a a mother load of crapware from Lenovo, and it gets really annoying with the loading times if you don't get it out.
Four, the sound is crappy but loud enough, like those radios you get to use in a bathroom. They don't distort but it's not great sound.

As you can see this negative features aren't really negative if you take in account that this is a business/student convertible to persons who are always and on the run traveling.

By the way the most delicious thing in the laptop is to navigate in the internet while in bed with the wireless internet on.
Tenho já dois rascunhos no forno depois da falta de combustível mental que existiu em 2007. Embora já tenha recomeçado a escrever tenho tido uma dificuldade muito grande em escrever duma forma estruturada e coerente como antes pensava conseguir escrever. Até este pequeno texto, escrito em forma de desabafo só para ver se os dedos desemperram, me custa brutalmente, com a agravante de me perder constantemente nos raciocínios. Os dois textos que tenho na calha são bastante diferentes um do outro. Um está em inglês e é sobre a minha recente campanha pessoal para ser piloto de linha aérea. O outro é muito semelhante ao género de textos que venho publicando no blog, desta feita sobre a origem do universo numa perspectiva divina. Está-se a mostrar bem mais complicado do que inicialmente tinha matutado na minha cabeça como habitualmente faço. As minhas ideias sobre a origem do Universo estão bastante mais dispersas e desorganizadas do que o normal dando origem a misturas de temas que estão muito próximos, mas que não são o objectivo do texto e dão origem a contradições que são difíceis de corrigir.
Enfim de uma maneira desajeitada vou quebrar o ano em branco aqui no blog, com promessas de esvaziar a minha cabeça, para que me possa focar no que realmente interessa, a minha futura carreira de piloto.

Saturday, November 25, 2006

Significado da Vida

Como acho que já todos se esqueceram do blog e como o deixei definitivamente de o publicitar todos os textos aqui passaram a ser como que uma especie de diário(anuario??) pessoal e privado, embora continue a existir, pesquisavel até, pelo google :D

O tema de que hoje venho escrever é sobre o significado da vida. Desde sempre, continuando ainda hoje, que o homem se questionou quanto ao seu propósito no mundo.
Eu pessoalmente, com 17 anos em 2006, penso que respondi á pergunta para a minha pessoa.
Eu como Ser Humano e vivo tenho como unico e exlusivo propósito reproduzir-me, A imortalidade. Não através da minha existência eterna sob a forma imutável do meu ser, mas sim como uma parte significativa dos traços físicos e consequentemente mentais de alguém meu descendente. Por isso a minha morte algum dia será irrelevante se eu me tiver reproduzido e produzido descendencia capaz de por sua vez se reproduzir num ciclo infinito.
A teoria evolucionista em palavras vagas é normalmente enunciada como a "sobrevivência do mais forte", omitindo o significado de sobrevivência neste contexto. Sobrevivencia significa opurtunidade de procriar, pois se um ser vivo estiver morto consequentemente as suas células sexuais também o estão. O amor, paixão odio etc são tudo vantagens que os seres mais fortes possuiram, permitindo-lhes reproduzir-se sem precisarem de justificação. "Paixão e pronto".
O homem distanciou-se relativamente aos outros animais devido á sua auto-consciencia. Isto é, se tinha fome, sabia que era porque não comia há algum tempo, o mesmo para a sede e outras sensações. Mas porque é que só tão tarde é que equacionámos as emoções mais profundas tal como o amor, na busca pelo significado da vida? Não sei, mas se tivesse sido alvo de maior atenção talvez hoje teríamos religiões em que o nascimento de alguém, quem quer que fosse, seria um pedaço de milagre, de divino, de paraíso. Porque haveriamos nós que ter como propósito da vida a idolatração de algo que não sabemos se existe e prática de rituais que não conhecemos? Um nascimento e posterior qualidade de vida até uma morte natural deveria ser o objectivo da vida num Deus, tornando-se presente de forma material todos os dias.
Reprodução é a resposta!

Sunday, March 26, 2006

Boas a todos. Já há algum tempo desde que aqui não escrevo, talvez porque o blog me libertou de tal maneira que me desinspirei..esvaziei. Bem, mas o que interessa é que tenho um novo pensamento na calha senão não me dava ao trabalho de criar um texto inútil.Desta feita o pensamento tem a ver com o doce gosto da maldade.
A maldade na minha opinião é uma coisa normalissíma embora a nossa sociedade e nós proprios nos censuremos em relação à sua natureza. Analizemos as coisas da seguinte maneira:
Todos nós já alguma vez nas nossas vidas desejámos mal a alguém ou sentimos uma chama bestial quando causamos sofrimento, fisico ou emocional propositadamente, mas raramente o expressamos publicamente por vergonha. Isto demonstra uma grande hipocrisia por parte dos nossos seres. Para demonstrar que o que aqui estou a dizer não é infundamentado dou-vos um de muitos exemplos. O caso da guerra e todas as atrocidades inerentes por mais legitima que nos pareça.
É ou não verdade que ela existiu desde as tribos recolectoras pre-históricas até aos nossos tempos?
É ou não verdade que todos ensinamos as nossas crianças a maldizer a guerra e que quando escrevemos, ou dizemos os nossos votos de ano novo, natal, etc... para um público de maior dimenção, afirmamos normalmente que gostaríamos que as guerras acabassem já amanha e que ouvesse paz?
É ou não verdade que as cinco grandes religiões do mundo têm como base uma mensagem de paz? ( No entanto muitas têm historial de guerras. Irónico.)
Então porque ainda existem guerras e não vivemos num mundo em que ninguém guerreia ninguém e onde há paz?
Porque nos nosso intimos há a animalidade de que vos falei, uma vontade de fazer o outro sofrer de maneira penosa mesmo que isso não traga nenhuma mais valia para ninguém.
O propósito deste texto?
Na minha opinião se percebermos as nossas imperfeições e defeitos poderemos criar um ponto de partida para criar salvaguardas que nos ajudem a controlar mas não repudiar nem silenciar os nossos instintos, pois esses mesmos instintos, sençasões ou vontades podem-nos enganar quando menos esperamos por eles e contornar e distorcer a nossa própria razão sem que nos apercebamos
Pensem nisto..