PIC Programmer using a USB to Serial Converter - LVP

Background

Nowadays, computers do not have serial/parallel ports. Neither has my laptop. Since my interest in PIC programming was temporary and for hobby purposes, i did not want to get a commercial USB based PIC programmer. Besides, where's the fun in that ? I decided to get a cheap USB-Serial converter, and see if it could make it work.

Important : This hack with the flip-flop is needed only if your USB-Serial converter does not support the BREAK functionality. If you want to buy a new one, i have been told that Cp2102 is a better option than pl2303.

Current Status

I used the Bafo Db9 USB-Serial converter(cost 200 Rs), and tried to program the PIC18F4550, and its a success. I had initially a problem reading the EEPROM, this was solved by a resistor between PGD and DTR. Its pretty slow, takes around 1 min to program a 350 byte hex file, verifying only 10 bytes instead of the entire code.

Disclaimer : This is an experiment, do it at your own risk.

PIC Programmer requirements

There are 2 modes for programming PICs :

1. HVP - High voltage programming in which the MCLR pin is raised above Vdd( typically to > 9V) to make the PIC enter into programming mode. 
2. LVP - Low voltage programming, in which the PIC can be programmed with the same voltage as VDD. Advantage is that high voltage is not needed, one can use the USB port power supply to program the PIC. Not all PICs support LVP, and LVP support can be turned on/off using a configuration setting, but this has to be done in HVP mode. There are 2 modes in LVP :

• PGM : A separate PIN, PGM is raised high, to VDD,  to make the PIC enter into programming mode. Disadvantage is a separate PIN blocked for this. My experiment is in LVP mode. 
• KEY : A special sequence of bytes fed into the PIC, to make it enter programming mode, usually with VPP held low.

My experiment is in the LVP-PGM mode. After the PIC enters the programming mode, we feed it commands/program-bytes on the DATA line( which is bidirectional on the PIC),  in sync with the clock signal provided on the clock line. This is Microchip's ICSP( In-circuit-serial-programming) protocol. For programming specs of various PICs, see http://www.microchip.com/TechDoc.aspx?type=Programming

Below is a (rather dirty)  diagram showing pins needed for PIC programming :

 

As we see 5 pins would be required of our programmer :

4 Outputs - MCLR, DATA_OUT, PGM, Clock

1 Inputs - DATA_IN

If our programmer is something with GPIOs, like a Raspberry Pi, then a GPIO pin can work as both input and output, so two separate pins will not be needed for DATA.

However, this post is going to be about using the USB to Serial Converter.

Take a look at http://www.db9-pinout.com/ to learn about a serial port pins.

Now, a serial port DB9 connector has only 3 lines which can be outputs :

RTS, DTR  and Txd. There are multiple candidates for input, usual one is CTS.

So we are one output pin short. However, as per research on the web, e.g. : http://artjoy.narod.ru/eng/index_eng.html, we see that the MCLR pin can simply be tied to VDD in LVP mode. So now, we seem to have met the pin requirements of the programmer. 

So if we build a simple JDM like programmer e.g. http://www.instructables.com/id/Simple-JDM-PIC-Programmer/, controlling PGM instead of MCLR from Txd, tie MCLR to VDD, and use some of the programming softwares available like PicPgm, it should work, right ?

Well, sadly, no.

Our usb-serial port is not a real serial port. It differs in two important respects :

1. Voltage levels : The Txd pin on a real serial port has low level voltage between 3-15 V, and a high level voltage between -3 to -15V. This is made use of in most serial programmer circuits to generate an HVP voltage. Whereas the usb-serial port usually has the TTL range of 0-5V. Well thats Ok you say, 5V is sufficient for LVP isn't it ? Yes, but there is another significant difference. In a real serial port, the Txd pin is held high in idle state, so it can provide a non-zero voltage in idle state, whereas in the usb-serial case, the txd pin is at zero-level in idle state, and goes high only when transmitting data.
2. Timing issues : USB is a scheduling based protocol, where data exchange occurs in time-frames controlled by the USB-controller. There is a limit on how much time can be allocated in a frame, to what kind of data(interrupt,bulk,isochronous). What this means is, if i want to toggle a line( e.g. RTS/DTR ) high or low for clock or data, it may not occur immediately. First the request is sent to USB controller. That will encode my request into a packet format. Maybe it will get sent in the next time frame, since the current one is full. Then the USB device at the other end will decrypt the packet, analyze my request, see that it means to raise the RTS line, and then execute it. Similar for a read. For full-speed USB normally used in such converters, frame time is 1 ms, and if say 20/30% of it is reserved for protocol handling/other kinds of data, it means that our frequency cannot be greater than 30% of 1ms. In practice, it can be even less, as we will see. Thus USB protocol is not really suited for real time applications.
3. Missing functionality. e.g. setting BREAKs. The Txd line of a real serial port is held high normally to show its active, and this can be brought low by transmitting all zeroes for some specific period, or indefinitely, by using the BREAK functionality. Usually, in a usb-serial adapter with 0-5v levels, the 0s have the opposite effect, of raising the line high. This is used by many of the programming softwares to manipulate the Txd line, and this can be missing in some USB-serial converters, like mine. See setbreak() in TERMIOS, EscapeCommFunction() in Windows  ioctl(handle, TIOCCBRK, 0) and ioctl(m_handle, TIOCSBRK, 0)) on Linux.

 Lets try to solve these issues :

Making the Txd Pin stay high for PGM

Why have we marked Txd for PGM ? Because we want exact and frequently changing 0-1 control on DATA and CLOCK pins, whereas Txd deals with bytes and not bits, and data is sent with start,stop,parity bits, so that even if we send 0, a 1 will also result due to the stop bit etc. PGM, on the other hand, just needs to be held high during LVP, and may be achieved more easily. Now lets see some options :

1. Using a Flip-flop that will change state when a pulse is received from Txd. The output is fed to PGM. This is the approach i have finalised. To set PGM, we send a '0' on  Txd, which results in a high pulse. Since its a toggle, next time we send the pulse, PGM will be reset. See the setPGMState() method in burnGPIO.py for how this is done. The schematic is at http://www.electronicshub.org/jk-flip-flop-using-cd4027/. NOTE : It worked best for me with just the 10k resistor(R3) connected from clock to GND. This circuit should be first tested with a push button, and once that works, try it with the Txd pin. Also, having a LED may load the output, make sure the output high voltage is above 4.5 volts. The LED should go high in LVP mode,( i.e when the program is executed) and stay low otherwise. Ensure that it is low before you run the program.
2. Using a simple latch as described here : http://homemadecircuitsandschematics.blogspot.in/2011/12/simple-and-useful-transistor-latch.html. So we turn on the latch to make PGM high at the start of the program. The disadvantage is that we cannot make it low programatically, it has to be done manually using a switch that connects the base to the ground. Another caveat is that the input should not go to ground after the pulse, but Txd is going to do so, so we prevent it by adding a diode in series before the base the the 1st transistor.
3. Since the BREAK functionality is missing in my usb-serial converter, another option is to continuously keeping Txd high by transmitting on it, in a separate thread. This does not work as is, probably because Txd does not rise fast/high enough for PGM. Does not work with driving a transistor either. The extra thread might also cause timing issues for the main operation. See the burnGPIO_txdOnThread.py for this.

Currently, option 1 seems the best one, and its my default.

The USB timing issue

We know that USB is going to introduce a delay, but we need to find exactly how much, so that we can use that for our clock operation. Thankfully, the clock rate for ICSP is not fixed, and can be varied as per the programmer.

What we do, is connect the DATA_OUT(RTS/DTR) to DATA_IN(CTS/DSR), output a 1 on DATA_OUT, and try to read it on DATA_IN, with a delay in between. Similarly for a 0 on DATA_OUT. And we adjust the delay until we can successfully read what we wrote. As per my tests, this delay turns out to be 5ms. See testSerial.py for this test.

The final hardware circuit

This is the final circuit used. The board on the right bottom contains the flip-flop circuit. It receives the Txd(3) line as the input, and its output is used to drive the PGM pin. RTS(7) is used as data-out, CTS(8) as data-in, DTR(4) as clock. VPP/MCLR is tied to VDD thru a pull-up-resistor of 10k. The crystal and capacitors connected to the bottom side of the PIC are not needed here, they are used when running the PIC as a USB device. The 3 transistors at the top-left are also not needed here, they are level shifters between the Raspberry Pi(3.3V) and the PIC(5V). i initially programmed the PIC using the Raspberry Pi, and they were used then. See http://electronics.stackexchange.com/questions/82104/single-transistor-level-up-shifter and http://elinux.org/RPi_GPIO_Interface_Circuits if interested.

Note : A resistor of 1-5K, between DTR and PGD is a must. It prevents a short between these two when PGD is in output mode and DTR is low, and ensures that a proper signal reaches CTS. Optionally, it can also be added to the RTS and CTS lines. Thanks to Darron from kewl.org for pointing this out.

Either of DTR/RTS can be used for PGD/PGC. My earlier schematic was using RTS for PGD, but i am reverting to DTR, since its more commonly used.

So now we have resolved these two issues. Next is, what software to use for programming ?

The Programming Software

Now we need a software to program the PIC. This software will read the .hex file we want to program into the PIC, and manipulate the clock, data. pgm lines to achieve this. There are many such softwares on the net, e.g. PicPgm, WinPic. Some allow a custom interface to be defined. However, none seem to handle the Txd line and delay like we want. Not being a C/C++ guy, I was looking for something in Java or a scripting language, that i could easily understand and modify. Then i came across a software in python - burnLVP, written for the Raspberry pi. See https://github.com/danjperron/burnLVP. Thanks Dan ! i was able to use it to program my PIC using the Rpi. Then i decided to see if i could make it work for USB-serial. There were references to the Rpi PINs in multiple places. I encapsulated all into the burnGPIO.py, which now represents the hardware interface. Currently, there are 2 versions of it, one for the USB-Serial and one for the Rpi. The one we want to use, should be copied as burnGPIO.py. The rest of the program remains the same. So if somebody want to use some other hardware interface for programming, they can create an appropriate version of burnGPIO.py and use that. Note that this software does not have a UI, its run from the command line as "python burnLVPx.py <hexfilename>". The intelhex library for python needs to be installed.

Due to the 5*2ms clock, the execution is quite slow. It takes a minute to burn

a small hex file of around 300 bytes. Since verification of the entire program memory would take a long time, i have introduced a parameter to those methods specifying how many bytes to check. Currently, i use 10. And for some reason i don't know yet, reading the EEPROM is not working. (always returns 0). So that is bypassed with a "True or" condition.

Another change in burnLVPx.py is the PIC families to be used. Currently i have kept only the one for the 4550. See the "AllCpuFamily" variable.

While the software has been named burnLVP, i believe that with minor modifications, it should work for HVP as well. The hardware will change, a >9V MCLR will be needed, and PGM is not needed, it should be tied to ground.

Update : i got HVP to work too. See my next post at http://chaukasalshi.blogspot.in/2015/01/pic-programmer-using-usb-to-serial.html

Sample output :

python burnLVPx.py Test.X.production.hex

File " Test.X.production.hex " loaded
Scan CPU
check  PIC18F2XXX ...
4610
Cpu Id   = 0x1200
Revision =  0x2
Found  PIC18F4550 from Cpu Family  PIC18F2XXX
Cpu Id: 0x1200  revision: 2
Bulk Erase  ..... Done!
Program code [ 32768 ] blank check.Passed!
Writing Program.Done!
Program check .Passed!
Writing EEPROM data[ 256 ]........Done!
Writing ID  ... Done!
ID Check . ... Passed!
CONFIG Burn  ... Done!
Config Check . ... Passed!
Program verification passed!

Trouble shooting

1. The flip-flop state should be off before programming, On during, and off again. Test this circuit separately first.
2. If you get the CpuTag as 0, most probably PGM is not being applied correctly.
3. PICs other than 4550 have not been tested, they may have issues.

In case somebody wants to do this in java, see https://code.google.com/p/java-simple-serial-connector/, which works nicely as a serial-port api.

Download the modified code

The modified code is available as burnLVP*.tar.gz file at http://milunsagle.in/webroot/downloads/or from the repository at https://github.com/danjperron/burnLVP/tree/usb2ser

Other open source programming softwares :

• Kewl : http://wiki.kewl.org/dokuwiki/interfaces. A nice, simple command line tool, easy to modify, supports many PICs
• WinPic : http://www.qsl.net/dl4yhf/winpicpr.html( No longer maintained )
• wxpic : a derivative of WinPic : http://wxpic.free.fr/