Sensitivity of Image Sensors to IR

It is probably common knowledge that the camera in your phone can see the LED on your TV remote go ON and OFF whereas your eye cannot. This happens because the LED on the remote emits light in the infra-red range to which the human eye is not sensitive. But most image sensors are sensitive to IR.

This can be a problem when you take pictures. You expect the image to look exactly what your eye see. But your camera can see more which may lead to unwanted effects on the image. So most cameras have a filter which reduce the effects of IR on the image. Most of the times, you can see the IR led in your remote glow only when you directly point it at the camera. This is because the filter greatly attenuates IR such that reflected light is very hard to see.

It may be pretty hard to guess where I am going with this. But before I get to the issue, there is another topic I want to shed some light on.

Most image sensors on the market make use of the Bayer pattern. The Bayer pattern is an arrangement of pixels sensitive to red, green and blue light in a particular order so that they combine to give a complete color image. So when you analyze an image, it contains three channels (3 two dimensional matrices), one for each color. If you take an image of a red object, it appears white in the red channel and black in the other channels. When I say it appears white, it just means that it has a high value close to 255 (for 8-bit images). Similarly, a blue object appears white on the blue channel and black on the other two channels.

Most people are content to verify that their camera can indeed see the IR led on a remote glow. But I went one step ahead and looked at the red, green and blue channels of the image. I was surprised by what I saw.

One would expect that the blue and green filters of the Bayer pattern would be able to eliminate the IR light. But what I saw was that the IR was consistently bright in all the three channels. Initially I thought that I had made a mistake and re-took the image and checked the code. But again I got the same result.

I haven’t been able to get a satisfactory explanation for this. Light is filtered according to their wavelengths in the Bayer pattern and by this line of thought even IR radiation must have been filtered out. What am I missing here? I would be glad to hear what you think of this.

Original Image
Original ImageOriginal
Blue Channel
Blue Channel
Green Channel
Green Channel
Red Channel
Red Channel

Virtual Machine for Arduino

Why a virtual machine? Firstly, I had a lot of time. Secondly, because it can be done. Also, it is fascinating to me that a device can keep its actual hardware hidden and project an entirely different hardware to the user. I have to say that I have not had any formal education on virtual machine. So I don’t know for sure if this fits into the ‘definition’ of a virtual machine but I’m certain that it can at least be called a quasi-virtual machine.

I have designed this for Arduino Mega and so had no memory issues. The basic firmware running on the Arduino is about 10kb. So Uno is out of the picture. But you can try to optimize the code and fit it into a Uno.

The Arduino is running a virtual stack machine and for me, the most interesting part is that you can run several of them simultaneously. I have tested with two of them running simultaneously but it is scalable and the code certainly supports that. One of the biggest challenges I faced was to device a scheme for multiple VMs to communicate. Yes there are plenty of schemes out there but I had to custom fit it into my machine and instruction set.

Also, considerable thought went into how the device I/O would work. I got the basic instruction set from Mr. Bruce Land’s page with his permission and then had to modify it to suit the machine I had in mind.

All the interaction with the VM is via UART. An important thing to note is that you cannot use the serial monitor that comes with the Arduino IDE since it sends ASCII values whereas the device expects the opcodes in hex values. I have been using a free terminal program for this.

This has been a real fun project to do. I have learnt a lot from it. Modifying the instruction set and implementing it has been very educational. I now appreciate the thoughts and effort that goes into designing an instruction set (Although I have designed a very small subset here) for a machine. Trying to get multiple machines running has been real fun. Though it doesn’t improve performance, running parallel machines has been a learning experience. I have done preliminary testing of all the instructions and they seem to work fine.

All VMs have their own private stack, program memory, instruction pointer, stack pointer and a couple of flags – output ready and execution done.

Instruction Set


 OpcodesDescriptionInstruction Size in bytesNop00No operation1Reap01Read from port1Wrip02Write into port1Jmpf A03Forward jump of instruction pointer

Ip = ip + A3Jmpb A04Backward jump of instruction pointer

Ip = ip – A3Jfz05Jump forward if zero

If top==0, ip = ip + A1Jbz06Jump backward if zero

If top==0, ip = ip – A1Jfnz07Jump forward if not zero

If top!=0, ip = ip + A1Jbnz08Jump backward if not zero

If top!=0, ip = ip – A1Load09Load instruction

Top = stack[top]1Store10Store instruction

Stack[next] = top1Add11Add instruction

Top = top + next1Sub12Subtract instruction

Top = next – top1Mul13Multiply instruction

Top = top * next1And14And instruction

Top = top && next1Or15Or instruction

Top = top || next1Band16Bitwise And instruction

Top = top & next1Bor17Bitwise Or instruction

Top = top | next1Bxor18Bitwise Xor instruction

Top = top ^ next1Eq19Equals

If top == next, top = 1 else top = 01Gt20Greater than

If next > top, top = 1 else top = 01Lt21Lesser than

If next < top, top = 1 else top = 01Ge22Greater than or equal to

If next >= top, top = 1 else top = 01Le23Lesser than or equal to

If next <= top, top = 1 else top = 01Ne24No equal to

If next != top, top = 1 else top = 01Drop25Drop the top of stack1Dup26Duplicate top of stack

Top two values on the stack are same1Over27Add next to top of stack

<next top next>1Rs28Right shift

Top = top >> 11Ls29Left shift

Top = top << 11Ed30Execution done

Sets ed flag1Sero31Serial out

Top of stack is sent via uart1Put A32Put A on stack

Top = A3Opr33Set output ready flag

Used in multi- VM scenarios1Pushc34Push the top of stack into top of common memory1Popc35Pop the top of common memory to the top of stack1Wopr B36Wait for VM B’s output to be ready

(For multiple VM scenario)2

* ‘A’ represents a 2byte integer and ‘B’ represents a single byte value

Default value for

  • Program memory – 2048 bytes
  • Stack size – 512 (integers)
  • Common memory – 64 (integers)

As you see in the opcode table, some instructions are 3 bytes and some are single byte instructions. When using a jump instruction, it is important to keep the size of the instruction in mind. The jump instruction will take relative values and the number given must be in terms of bytes and not in terms of no. of instructions.

There are two recommended ways to end the program in this VM. The first way is to use the ‘ed’ instruction. This sets a flag and the VM stops execution. The second way is to have a ‘nop’ followed by a jump to the nop which in effect is an infinite loop.

Though most of the instructions are self-explanatory, I believe that some instructions do require me to explain them.

  • Reap: Read port

The firmware has been setup such that Port A and Port B are inputs and they are mapped to the stack at the fourth last and third last addresses respectively. The reap instruction will load the value on port A to stack[STACK_SIZE – 4] and value on port B to stack[STACK_SIZE – 3]. After that, a put A and load instruction can be used to bring the required port value to the top of stack.

  • Wrip: Write port

Port C and port D are the output ports and they are mapped to the stack at the second last and last addresses respectively. Wrip instruction will load the value in stack[STACK_SIZE – 2] to port C and value in stack[STACK_SIZE – 1] to port D. the stack is 16 bit wide and so, the lower 8-bits will be put onto the port.

  • Ed: execution done

Execution done instruction sets a flag and the VM stops running.

  • Sero: Serial Output

This instruction takes the top of the stack and sends it out serially via the UART. The data at the top of the stack is now lost.

  • Opr: Output ready

Sets the output ready flag of the VM. Indication for other VMs that may be waiting for the output of this VM to be ready.

  • Wopr B: Wait for Output to be Ready

Wait for the output of VM B to be ready. Execution continues once VM B indicates that its output is ready by setting the output ready flag.


If you reset the Arduino board, the firmware is reset and all data and programs of the VM are lost. So there must be some way of resetting the VM without resetting the firmware. Initially, I pondered over the thought of writing the VM program onto an EEPROM but decided against it. Pin 49 of the Arduino board is the reset for the VM. On pulling this pin low, you get a serial prompt asking for which VM to reset. It is important to note that all VMs will stop execution until a particular VM has been reset. Also, resetting will not erase will erase everything but the program of the VM.


Binary Instructions

All binary instructions (instructions requiring two operands such as add or and) operate on top of stack and the second value on stack. The output value is obtained on the top of stack. The two operands are lost and stack pointer is decremented by one.

Common Memory

Communication between the VMs takes place through the common memory. This a small stack which can be accessed by all VMs. Like a normal stack, a pop instruction will cause the stack pointer to jump and if another VM needs the same value, it must be pushed onto the stack again.

Programming Mode

Say multiple VMs are running. And you want to re-program just one of them. There is a solution for that. Pulling Pin 48 low puts the device into programming mode and allows a single VM to be reprogrammed. The downside is that all other VMs stop execution for that duration.

What Next

So what next? Well, the machine does need an assembler and maybe a compiler. But it is certainly not worth the effort as long as I am the only one using it. Also, I did give some thought to adding interrupts but decided to drop it. So maybe that will come up next. But maybe I need to write some code that will help in programming the VM. The current method goes back to the days of manually writing the opcodes in hex.

This may seem to be very abstract to you and if you need any clarifications or have any comments or suggestions or ideas, feel free to comment and I can assure you that you will get a reply.

Source code :