Sensitivity of Image Sensors to IR

It is probably common knowledge that the camera in your phone can see the LED on your TV remote go ON and OFF whereas your eye cannot. This happens because the LED on the remote emits light in the infra-red range to which the human eye is not sensitive. But most image sensors are sensitive to IR.

This can be a problem when you take pictures. You expect the image to look exactly what your eye see. But your camera can see more which may lead to unwanted effects on the image. So most cameras have a filter which reduce the effects of IR on the image. Most of the times, you can see the IR led in your remote glow only when you directly point it at the camera. This is because the filter greatly attenuates IR such that reflected light is very hard to see.

It may be pretty hard to guess where I am going with this. But before I get to the issue, there is another topic I want to shed some light on.

Most image sensors on the market make use of the Bayer pattern. The Bayer pattern is an arrangement of pixels sensitive to red, green and blue light in a particular order so that they combine to give a complete color image. So when you analyze an image, it contains three channels (3 two dimensional matrices), one for each color. If you take an image of a red object, it appears white in the red channel and black in the other channels. When I say it appears white, it just means that it has a high value close to 255 (for 8-bit images). Similarly, a blue object appears white on the blue channel and black on the other two channels.

Most people are content to verify that their camera can indeed see the IR led on a remote glow. But I went one step ahead and looked at the red, green and blue channels of the image. I was surprised by what I saw.

One would expect that the blue and green filters of the Bayer pattern would be able to eliminate the IR light. But what I saw was that the IR was consistently bright in all the three channels. Initially I thought that I had made a mistake and re-took the image and checked the code. But again I got the same result.

I haven’t been able to get a satisfactory explanation for this. Light is filtered according to their wavelengths in the Bayer pattern and by this line of thought even IR radiation must have been filtered out. What am I missing here? I would be glad to hear what you think of this.

Original ImageOriginal

Blue Channel

Green Channel

Red Channel

Virtual Machine for Arduino

Why a virtual machine? Firstly, I had a lot of time. Secondly, because it can be done. Also, it is fascinating to me that a device can keep its actual hardware hidden and project an entirely different hardware to the user. I have to say that I have not had any formal education on virtual machine. So I don’t know for sure if this fits into the ‘definition’ of a virtual machine but I’m certain that it can at least be called a quasi-virtual machine.

I have designed this for Arduino Mega and so had no memory issues. The basic firmware running on the Arduino is about 10kb. So Uno is out of the picture. But you can try to optimize the code and fit it into a Uno.

The Arduino is running a virtual stack machine and for me, the most interesting part is that you can run several of them simultaneously. I have tested with two of them running simultaneously but it is scalable and the code certainly supports that. One of the biggest challenges I faced was to device a scheme for multiple VMs to communicate. Yes there are plenty of schemes out there but I had to custom fit it into my machine and instruction set.

Also, considerable thought went into how the device I/O would work. I got the basic instruction set from Mr. Bruce Land’s page with his permission and then had to modify it to suit the machine I had in mind.

All the interaction with the VM is via UART. An important thing to note is that you cannot use the serial monitor that comes with the Arduino IDE since it sends ASCII values whereas the device expects the opcodes in hex values. I have been using a free terminal program for this.

This has been a real fun project to do. I have learnt a lot from it. Modifying the instruction set and implementing it has been very educational. I now appreciate the thoughts and effort that goes into designing an instruction set (Although I have designed a very small subset here) for a machine. Trying to get multiple machines running has been real fun. Though it doesn’t improve performance, running parallel machines has been a learning experience. I have done preliminary testing of all the instructions and they seem to work fine.

All VMs have their own private stack, program memory, instruction pointer, stack pointer and a couple of flags – output ready and execution done.

Instruction Set

Mnemonic

 OpcodesDescriptionInstruction Size in bytesNop00No operation1Reap01Read from port1Wrip02Write into port1Jmpf A03Forward jump of instruction pointer

Ip = ip + A3Jmpb A04Backward jump of instruction pointer

Ip = ip – A3Jfz05Jump forward if zero

If top==0, ip = ip + A1Jbz06Jump backward if zero

If top==0, ip = ip – A1Jfnz07Jump forward if not zero

If top!=0, ip = ip + A1Jbnz08Jump backward if not zero

If top!=0, ip = ip – A1Load09Load instruction

Top = stack[top]1Store10Store instruction

Stack[next] = top1Add11Add instruction

Top = top + next1Sub12Subtract instruction

Top = next – top1Mul13Multiply instruction

Top = top * next1And14And instruction

Top = top && next1Or15Or instruction

Top = top || next1Band16Bitwise And instruction

Top = top & next1Bor17Bitwise Or instruction

Top = top | next1Bxor18Bitwise Xor instruction

Top = top ^ next1Eq19Equals

If top == next, top = 1 else top = 01Gt20Greater than

If next > top, top = 1 else top = 01Lt21Lesser than

If next < top, top = 1 else top = 01Ge22Greater than or equal to

If next >= top, top = 1 else top = 01Le23Lesser than or equal to

If next <= top, top = 1 else top = 01Ne24No equal to

If next != top, top = 1 else top = 01Drop25Drop the top of stack1Dup26Duplicate top of stack

Top two values on the stack are same1Over27Add next to top of stack

<next top next>1Rs28Right shift

Top = top >> 11Ls29Left shift

Top = top << 11Ed30Execution done

Sets ed flag1Sero31Serial out

Top of stack is sent via uart1Put A32Put A on stack

Top = A3Opr33Set output ready flag

Used in multi- VM scenarios1Pushc34Push the top of stack into top of common memory1Popc35Pop the top of common memory to the top of stack1Wopr B36Wait for VM B’s output to be ready

(For multiple VM scenario)2

* ‘A’ represents a 2byte integer and ‘B’ represents a single byte value

Default value for

• Program memory – 2048 bytes
• Stack size – 512 (integers)
• Common memory – 64 (integers)

As you see in the opcode table, some instructions are 3 bytes and some are single byte instructions. When using a jump instruction, it is important to keep the size of the instruction in mind. The jump instruction will take relative values and the number given must be in terms of bytes and not in terms of no. of instructions.

There are two recommended ways to end the program in this VM. The first way is to use the ‘ed’ instruction. This sets a flag and the VM stops execution. The second way is to have a ‘nop’ followed by a jump to the nop which in effect is an infinite loop.

Though most of the instructions are self-explanatory, I believe that some instructions do require me to explain them.

• Reap: Read port

The firmware has been setup such that Port A and Port B are inputs and they are mapped to the stack at the fourth last and third last addresses respectively. The reap instruction will load the value on port A to stack[STACK_SIZE – 4] and value on port B to stack[STACK_SIZE – 3]. After that, a put A and load instruction can be used to bring the required port value to the top of stack.

• Wrip: Write port

Port C and port D are the output ports and they are mapped to the stack at the second last and last addresses respectively. Wrip instruction will load the value in stack[STACK_SIZE – 2] to port C and value in stack[STACK_SIZE – 1] to port D. the stack is 16 bit wide and so, the lower 8-bits will be put onto the port.

• Ed: execution done

Execution done instruction sets a flag and the VM stops running.

• Sero: Serial Output

This instruction takes the top of the stack and sends it out serially via the UART. The data at the top of the stack is now lost.

• Opr: Output ready

Sets the output ready flag of the VM. Indication for other VMs that may be waiting for the output of this VM to be ready.

• Wopr B: Wait for Output to be Ready

Wait for the output of VM B to be ready. Execution continues once VM B indicates that its output is ready by setting the output ready flag.

Reset

If you reset the Arduino board, the firmware is reset and all data and programs of the VM are lost. So there must be some way of resetting the VM without resetting the firmware. Initially, I pondered over the thought of writing the VM program onto an EEPROM but decided against it. Pin 49 of the Arduino board is the reset for the VM. On pulling this pin low, you get a serial prompt asking for which VM to reset. It is important to note that all VMs will stop execution until a particular VM has been reset. Also, resetting will not erase will erase everything but the program of the VM.

 

Binary Instructions

All binary instructions (instructions requiring two operands such as add or and) operate on top of stack and the second value on stack. The output value is obtained on the top of stack. The two operands are lost and stack pointer is decremented by one.

Common Memory

Communication between the VMs takes place through the common memory. This a small stack which can be accessed by all VMs. Like a normal stack, a pop instruction will cause the stack pointer to jump and if another VM needs the same value, it must be pushed onto the stack again.

Programming Mode

Say multiple VMs are running. And you want to re-program just one of them. There is a solution for that. Pulling Pin 48 low puts the device into programming mode and allows a single VM to be reprogrammed. The downside is that all other VMs stop execution for that duration.

What Next

So what next? Well, the machine does need an assembler and maybe a compiler. But it is certainly not worth the effort as long as I am the only one using it. Also, I did give some thought to adding interrupts but decided to drop it. So maybe that will come up next. But maybe I need to write some code that will help in programming the VM. The current method goes back to the days of manually writing the opcodes in hex.

This may seem to be very abstract to you and if you need any clarifications or have any comments or suggestions or ideas, feel free to comment and I can assure you that you will get a reply.

Source code : https://github.com/nakul13/Arduino-Virtual-Machine

Arduino Video Game

I have always wanted to build a video game and finally, I have done it in my own little way.

I knew I wanted to build a space ship kind of shooter game. Two player ofcourse. Another thing I knew was that I wanted to use interrupts for all the button press – forward, backward, right, left and shoot. There were a lot of design decisions to be made in this project. The first one was what should be the shape of the spaceship and the bullet/missile. I tried out a few different choices and rounded in on one.

Simultaneously, I started looking for some interrupt libraries. There are several available actually. After trying out a couple, I decided to go with ‘pinchangeint-v2.19beta’.

First, I tried to get a spaceship on the screen shooting continuous bullets which worked just fine. And I forgot to mention this earlier, but I am using a 16×2 character LCD display because, well, I have no other option. I have a few spare 16×2 lcd lying around and I thought it would be a good idea to use them. I do agree that it makes the game very restricted, but it I went ahead with it anyhow.

I had to come up with an overall scheme of things. How would everything work together? As I mentioned earlier, the button press will be handled by interrupts. The most important thing to decide would be to decide how to update the screen. There is a main function which will draw all the characters on the lcd. The other important function is to detect collisions between ships, missiles and obstructions. And there are a bunch of other functions to calculate positions of missiles and ships.

When you look at the code you will find that it is kind of messy. I could have made it neater and organized it better. The functions are written separately for the two players. Instead, I could have made a single function and then pass some parameter into the function. I hope you get the idea. In my defense I would say that I had no incentive to do any of that. I did a quick and dirty implementation and if it works I am happy with it.

Here is a few pictures of the hardware and the game in action.

Video Game Spaceship Character

Different options for Spaceship Charecter

Video Game Borard

Game in Action

 

JK Megamini – Arduino 2560 Clone

 

Note from the video that the obstacles/barriers are designed to stop only the ship and not the missiles.

The source code is available at https://github.com/nakul13/Arduino-Video-Game.

Cryptanalysis

Electronics Mail

 

All of us have heard of cryptography. We are familiar with several algorithms. But how many of us have tried to break those algorithms? Have we ever put ourselves in the place of an attacker? Have we tried to extract data without knowledge of the key? In this article you will acquire a basic idea about the different techniques used and the challenges faced in breaking an algorithm.
Wikipedia defines cryptanalysis as the art and science of analyzing information systems in order to study the hidden aspects of the systems. The aim is to be able to extract the data from encrypted systems without the knowledge of the key. Cryptanalysis has proved to be a very important tool throughout history including the two world wars.
One of the simplest categories of ciphers is the substitution ciphers. Let us see how mono-alphabetic ciphers are broken. In all languages, certain letters…

View original post 532 more words

Reverse Engineering of Chips

Electronics Mail

Author: Nakul Rao I

 

We have all studied VLSI, or atleast we will. So we know how chips are made, how integrated circuits are constructed. But now there is a new fast emerging field; that of reverse engineering integrated chips. Basically it’s the same as opening up your old toys to find out what exactly is inside it, how does it work, how does it compare to others. The same idea is applied to chips, but the entire process is much more complicated.

The first question that may pop into your mind is why anyone would do it. There are several answers to that question.

The most important reason is cryptography. A lot of hardware around you is responsible for the safe keeping of information. Mobile phones, smart cards, RFID tags, digital set top box and even your car keys have some aspect of cryptography involved in them at…

View original post 783 more words