Using the freeRTOS PIC18 port. MPLAB C18 memory model set for: Small code model, large data model, multi-bank model. I.e. as per freeRTOS demos.

I have discovered what I think is a problem with the linker script used with the PIC port of freeRTOS.

This only showed up recently, despite the fact that I have been using freeRTOS for many weeks. I discovered that by declaring an arbitrary char array of a particular length, I could make my application fail to run correctly. Investigation revealed that variables defined in tasks.c were not being correctly addressed by the code generated by the C18 compiler. This was only a problem if task.c defined variables crossed a bank boundary, hence the dependence on the size of my char array.

The RAM data defined in a C source file is placed in a section specific to the source file. I.e. the RAM data defined in tasks.c is placed in either idata_tasks.o or udata_tasks.o The linker will place this in one of the data memory regions defined in the linker script. The linker ensures that a section will never cross a region boundary, they will always be placed in just one region. However, it is a requirement of the C18 compiler that every section is located within a single bank. If there is a region that spans more than one bank, it is possible that the section placed in this region will end up crossing a bank boundary. This causes the compiler generated variable access code to fail. The following is an extract from the MPLAB linker manual:

You must not combine data memory regions when using MPLINK Linker with MPLAB C18 C compiler. MPLAB C18 requires that any section be located within a single bank.

As the freeRTOS linker script defines one large area crossing multiple banks, the linker will place sections within this area, without restraining them to separate banks.

The default freeRTOS heap size is 1024 bytes long, leaving just two banks for variables. I suppose it is therefore unlikely that a section will finish up crossing a bank boundary. I had reduced the heap size to make more room for global variables. Perhaps this is why this problem appears not to have been experienced by others.

My fix, which admittedly lacks elegance, is as follows:

Mod the heap_1.c file to place the heap in a section named heap_section.
I.e.
#pragma udata heap_section

static struct xRTOS_HEAP
{
unsigned portLONG ulDummy;
unsigned portCHAR ucHeap[ portTOTAL_HEAP_SIZE ];
} xHeap;

#pragma udata

Change the linker script to:

LIBPATH .

FILES c018i.o
FILES clib.lib
FILES p18f252.lib

CODEPAGE NAME=vectors START=0x0 END=0x39 PROTECTED
CODEPAGE NAME=page START=0x3A END=0x7DBF
CODEPAGE NAME=debug START=0x7DC0 END=0x7FFF PROTECTED
CODEPAGE NAME=idlocs START=0x200000 END=0x200007 PROTECTED
CODEPAGE NAME=config START=0x300000 END=0x30000D PROTECTED
CODEPAGE NAME=devid START=0x3FFFFE END=0x3FFFFF PROTECTED
CODEPAGE NAME=eedata START=0xF00000 END=0xF000FF PROTECTED

ACCESSBANK NAME=accessram START=0x0 END=0x7F
DATABANK NAME=gpr0 START=0x80 END=0xFF
DATABANK NAME=heapregion START=0x100 END=0x3FF PROTECTED
DATABANK NAME=gpr4 START=0x400 END=0x4FF
DATABANK NAME=gpr5 START=0x500 END=0x5F3
DATABANK NAME=dbgspr START=0x5F4 END=0x5FF PROTECTED
ACCESSBANK NAME=accesssfr START=0xF80 END=0xFFF PROTECTED

SECTION NAME=CONFIG ROM=config

SECTION NAME=heap_section RAM=heapregion
STACK SIZE=0x60 RAM=gpr4

If the heap size exceeds the ‘heap_region’ defined in the linker script, the linker will complain. However, it is possible to waste precious RAM by defining the heap to be significantly smaller than ‘heap_region’. It is therefore advisable to ensure that the size of heap_region is only slightly larger than the actual heap size. I would appreciate learning of a more elegant solution.

As all heap allocated data is accessed via pointers, I’m assuming that there is no bank problem here, but would like confirmation.

Regards,
John Franklin

Thanks for the detailed feedback.

This is something that needs consideration for V3?

Hi John,

I think my problem is also related to this limitation.
In fact I think the problem is far more serious than what you describe.

The linker and the C18 compiler assume that all RAM sections will fit inside a memory region.

This means that
1) you cannot define more than 256 bytes of global/static variables inside one source file since 1 file will result in one section idata_SourceFileName.c. Maybe the compiler (not the linker) detects this and generates multiple datasections for 1 source file, but I'm not counting on that.

2) if the compiler is smart and generates multiple datasections for 1 source file: arrays still cannot be longer than 256 bytes, since the compiler will not set the BSR accordingly

3) I guess it is not possible with PIC to have a software stack of more than 256 bytes, so the number of local variables is also limited.


How does this apply to FreeRTOS?

The memory allocation function is not aware of the bank limitation of the PIC, thus it will just place a variable at the first available RAM position.

your fix prevents global variables to be allocated inside the 'bigchunk' memory (where they can cross a bank boundary) and forces them in one of the banked sections. This also forces the start of the heap back to the start of a bank. (Before, the heap would be placed after some globals variables, isn't it?) This is how I think your fix worked: the task.c variables are moved back down into the bank. However, this doesn't fix problems inside the heap!

I think 2 problems are to be expected.
FreeRTOS should ensure that the software stack for each task is located inside 1 memory bank.
semi-global variables such as Queues should be located inside 1 memory bank.


Now, my problem is the following:
I am using the PIC18F8525 to communicate with a magnet-card reader.

The data from the RS232 is put in a queue at 38000baud, the tick is 1kHz, so I receive about 4 bytes per tick
The queue is 80 bytes long.
To read the queue, I wanted to use something like this:


while ((cSerial2GetChar( RS232port, &cCardReaderReceivedByte, 10 )))// wait for response
{// Get response
responsestring[CardReaderLocalIndex++]=cCardReaderReceivedByte;
}

Which doesn't work. It just seems to exit the loop while there is still data in the queue. Maybe the task receives a resume signal during the waiting period? If only I knew from where.


The alternative routine works fine.

do
{// Get response
if (cSerial2GetChar( RS232port, &cCardReaderReceivedByte, 10 ))
{
responsestring[CardReaderLocalIndex++]=cCardReaderReceivedByte;
}
vTaskDelay(2);
}while (ucQueueMessagesWaiting( xRS232RxedChars)>0 );

I do not understand why. The first routine should loop until there has been no data for 10 ticks. The second monitors the state of the queue after just 2 ticks and actually gets all the data.

My impression for some weeks now has been that there is a memory problem. The queue array is 80 bytes long, so there is about 30% chance of a bank problem.

Another problem is that semaphores sometimes do not seem to block other tasks. Since a semaphore is a queue, I think this problem is related to the first.

After your posting I feel much stronger about this.

I'd like to hear your (or Richard's) answer to this.

Thanks

Paul van der Hulst

Hi Paul.

I’ll try and address your points, as best I can.

I’ve indented your text with a few underscores.

___The linker and the C18 compiler assume that all RAM sections will fit inside a memory
___region.
___This means that
___1) you cannot define more than 256 bytes of global/static variables inside one source file
___since 1 file will result in one section idata_SourceFileName.c. Maybe the compiler (not
___the linker) detects this and generates multiple datasections for 1 source file, but I'm not
___counting on that.

It is possible to create variables larger than a single bank. This is documented in a Microchip pdf file.
You can get it from this link: http://ww1.microchip.com/downloads/en/DeviceDoc/51295E.pdf

__2) if the compiler is smart and generates multiple data sections for 1 source file: arrays still
___cannot be longer than 256 bytes, since the compiler will not set the BSR accordingly

Arrays which span multiple banks have to be accessed in a special way, see the above pdf.
This raised some concern about the way the heap was allocated in freeRTOS. However, I’ve examined the compiler
generated code and all seems fine.

___3) I guess it is not possible with PIC to have a software stack of more than 256 bytes,
___so the number of local variables is also limited.

The normal C18 stack can be greater than 256 bytes, select the C18 large memory model.
In this case the BSR is not used for stack access. Stacks in freeRTOS are allocated on the heap
and accessed without using the BSR, so size should not present a problem.

___The memory allocation function is not aware of the bank limitation of the PIC, thus it will
___just place a variable at the first available RAM position.

By default all global/static variables are placed in the section allocated to the source file
that they are defined in. These will be mapped to one, and only one, of the regions defined in the linker
script. Provided each linker script defined region does not span a bank, all is well. The compiler
generated code will set the BSR to the relevant bank.

___your fix prevents global variables to be allocated inside the 'bigchunk' memory (where
___they can cross a bank boundary) and forces them in one of the banked sections. This
___also forces the start of the heap back to the start of a bank. (Before, the heap would be
___placed after some globals variables, isn't it?) This is how I think your fix worked: the
___task.c variables are moved back down into the bank. However, this doesn't fix
__problems inside the heap!

Not sure I follow this. It’s true that nothing other than the heap can now occupy the heap_region.
If, for example, your heap size is a lot less than heap_region, then the remaining space in heap region is
wasted, the compiler will not use it for global variables. All global/static variables will be placed in one
of the remaining linker script defined memory regions. You can examine the .map file to verify the placing of
all variables, including the heap.

__I think 2 problems are to be expected.
__FreeRTOS should ensure that the software stack for each task is located inside 1
__memory bank.
__semi-global variables such as Queues should be located inside 1 memory bank.

As memory allocated on the heap, e.g. task stacks, queues etc, are accessed via pointers the compiler will
not use the BSR. It is therefore irrelevant which bank they happen to be in. This certainly seems to be the
case. I’ve looked at the C18 generated code for signs that the BSR is being used to access the heap,
all seems OK. I would, however, feel happier to have this confirmed.

__The data from the RS232 is put in a queue at 38000baud, the tick is 1kHz, so I receive
__about 4 bytes per tick
__The queue is 80 bytes long.
__To read the queue, I wanted to use something like this:
__while ((cSerial2GetChar( RS232port, &cCardReaderReceivedByte, 10 ))) // wait for
__response
__{ // Get response
__responsestring[CardReaderLocalIndex++]=cCardReaderReceivedByte;
__}

__Which doesn't work. It just seems to exit the loop while there is still data in the queue.
__Maybe the task receives a resume signal during the waiting period? If only I knew from
__where.

I had similar problems with an earlier version of the C18 compiler. The C18 code generated
for pushing function parameters onto the stack was faulty. See my previous post. Changing to the latest
version cured this, as could be seen by examining the compiler generated code.

The behaviour resulting from variables crossing banks is many and varied, including erratic
queue behaviour. I was at first mystified why small, apparently irrelevant, changes to my code
could appear to fix the problem. I eventually traced this to the fact that the change had now pushed a
variable, that was previously straddling two banks, into just one bank. This is of-course, the whole
purpose of the changes to the linker script, to ensure that variables cannot cross a bank boundary.
Just to re-iterate, variables which are allocated on the heap (e.g. queues), are not accessed by using
the BSR, so can cross bank regions. At least, as far as I can ascertain, this seems to be the case.

Have you made the changes I proposed in my post, and are still having problems?

I hope this is of some use.

Regards,
John Franklin

The memory allocation functions in the download are intended for demonstration and will not always be suitable.

Does the pic compiler come with a malloc() library function? If so then presumably it must take into account the banking issues and could be used in place of the heap2 function. The heap3 function basically uses mallocs within critical regions. This is fine provided you don't want to malloc at real time/run time where the malloc() function would most likely take too long to complete.

Hi John,

As you pointed out: most of my points concerning banks and the BSR were wrong,
I had forgotten all about indirect addressing...
I spent most of the day stepping through the code and I now think the page thing is not an issue for data on the heap.
If it´s worth anything: I also am pretty sure the C18 generated code is ok.

By the way, I am using the newest versions of both FreeRTOS (2.6.1) and C18 (2.42) and have changed the code along the lines you proposed in your post.
The heap is now the only data object in that section (and protected), all other sections span only 1 bank.

Also my mathdata and .tmpdata are 0x14 and 0x04 long, so I changed post.c accordingly.

Still, the queue readout and semaphore problems persists.
Since taking a semaphore is the same (by a #define) as receiving something from a queue, this will be my next focus.

To Mr. Nobody:

malloc is not available by default. Microchip appears to have a simple ´SRAM memory manager in ´c´ ´ available, but this only allocates blocks up to 126 bytes.
Since this does not seem to be the source of the problem right now, I will leave it at that.

Greetings,

Paul van der Hulst

Hi Paul.
Thanks for the feedback. I would be interested in knowing what you discover. Good luck.

Regards,
John Franklin

Hi,

I finally tracked down the problem.

As you remember I had problems with serial i/o

The situation: An ISR sends data to the queue (at 38000 baud)
The communication task reads the data from the queue through the intermediate function cSerial2GetChar.

Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE
without having been blocked.

I checked this by keeping track of the TickCount before entering the function and after returning.

The problem lies in the generated assembly code. I included the C code and the assembly section below:

-----------------------------------------------------------------------------------
if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) )
{
return ( portCHAR ) pdTRUE;
}


14291 6FA4 ECEE CALL cQueueReceive, 0
14292 6FA6 F023 NOP
14293 6FA8 6EF5 MOVWF TABLAT, ACCESS
14294 6FAA 0E06 MOVLW 0x6
14295 6FAC 5CE1 SUBWF FSR1L, W, ACCESS
14296 6FAE E202 BC 0x6fb4
14297 6FB0 6AE1 CLRF FSR1L, ACCESS
14298 6FB2 52E5 MOVF POSTDEC1, F, ACCESS
14299 6FB4 6EE1 MOVWF FSR1L, ACCESS
14300 6FB6 50F5 MOVF TABLAT, W, ACCESS
14301 6FB8 0900 IORLW 0
14302 6FBA E003 BZ 0x6fc2

-----------------------------------------------------------------------------------
After returning from the call, WREG is copied to TABLAT (line 14293)
In line 14300 TABLAT is copied back to WREG and the result is tested (the 'if' condition).

Apparently, with the high interruptfrequency I'm using, one of the interrupt functions is modifying TABLAT, whigh results in the wrong 'if' evaluation.

I'm still trying to find out what the intermediate code is for,
I have not been able to reproduce this.
It probably has something to do with the function parameters being pointers or something like that.
Functions which only return a 'char' value and have no or only one char parameter produce the code:

call functionname,0
nop
iorlw 1
bz 0xxxxx <-- destination address


I fixed this by including TABLAT in the RS232 reception ISR function

#pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata")
void vSerial2RxISR( void )
{
static portCHAR cChar,cWoken;

Now communication is flawless.
I guess I have to do the same thing with the other ISR functions.

Greetings

Paul van der Hulst

Hi,

I finally tracked down the problem.

As you remember I had problems with serial i/o

The situation: An ISR sends data to the queue (at 38000 baud)
The communication task reads the data from the queue through the intermediate function cSerial2GetChar.

Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE
without having been blocked.

I checked this by keeping track of the TickCount before entering the function and after returning.

The problem lies in the generated assembly code. I included the C code and the assembly section below:

-----------------------------------------------------------------------------------
if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) )
{
return ( portCHAR ) pdTRUE;
}


14291 6FA4 ECEE CALL cQueueReceive, 0
14292 6FA6 F023 NOP
14293 6FA8 6EF5 MOVWF TABLAT, ACCESS
14294 6FAA 0E06 MOVLW 0x6
14295 6FAC 5CE1 SUBWF FSR1L, W, ACCESS
14296 6FAE E202 BC 0x6fb4
14297 6FB0 6AE1 CLRF FSR1L, ACCESS
14298 6FB2 52E5 MOVF POSTDEC1, F, ACCESS
14299 6FB4 6EE1 MOVWF FSR1L, ACCESS
14300 6FB6 50F5 MOVF TABLAT, W, ACCESS
14301 6FB8 0900 IORLW 0
14302 6FBA E003 BZ 0x6fc2

-----------------------------------------------------------------------------------
After returning from the call, WREG is copied to TABLAT (line 14293)
In line 14300 TABLAT is copied back to WREG and the result is tested (the 'if' condition).

Apparently TABLAT is changed between returning from the call and the evaluation at line 14301.
This can be either a task-switch or an ISR. In this case it is an ISR, ISRs don't save TABLAT by default.

I'm still trying to find out what the intermediate code is for,
I have not been able to reproduce this.

It probably has something to do with the function parameters being pointers or something like that.
Functions which only return a 'char' value and have no or only one char parameter produce the code:

call functionname,0
nop
iorlw 1
bz 0xxxxx <-- destination address


I fixed this by including TABLAT in the RS232 reception ISR function

#pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata")
void vSerial2RxISR( void )
{
static portCHAR cChar,cWoken;

Now communication is flawless.
I guess I have to do the same thing with the other ISR functions.

Greetings

Paul van der Hulst

Hi,

I finally tracked down the problem.

As you remember I had problems with serial i/o

The situation: An ISR sends data to the queue (at 38000 baud)
The communication task reads the data from the queue through the intermediate function cSerial2GetChar.

Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE
without having been blocked.

I checked this by keeping track of the TickCount before entering the function and after returning.

The problem lies in the generated assembly code. I included the C code and the assembly section below:

-----------------------------------------------------------------------------------
if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) )
{
return ( portCHAR ) pdTRUE;
}


14291 6FA4 ECEE CALL cQueueReceive, 0
14292 6FA6 F023 NOP
14293 6FA8 6EF5 MOVWF TABLAT, ACCESS
14294 6FAA 0E06 MOVLW 0x6
14295 6FAC 5CE1 SUBWF FSR1L, W, ACCESS
14296 6FAE E202 BC 0x6fb4
14297 6FB0 6AE1 CLRF FSR1L, ACCESS
14298 6FB2 52E5 MOVF POSTDEC1, F, ACCESS
14299 6FB4 6EE1 MOVWF FSR1L, ACCESS
14300 6FB6 50F5 MOVF TABLAT, W, ACCESS
14301 6FB8 0900 IORLW 0
14302 6FBA E003 BZ 0x6fc2

-----------------------------------------------------------------------------------
After returning from the call, WREG is copied to TABLAT (line 14293)
In line 14300 TABLAT is copied back to WREG and the result is tested (the 'if' condition).

Apparently TABLAT is changed between returning from the call and the evaluation at line 14301.
This can be either a task-switch or an ISR. In this case it is an ISR, ISRs don't save TABLAT by default.

I'm still trying to find out what the intermediate code is for,
I have not been able to reproduce this.

It probably has something to do with the function parameters being pointers or something like that.
Functions which only return a 'char' value and have no or only one char parameter produce the code:

call functionname,0
nop
iorlw 1
bz 0xxxxx <-- destination address


I fixed this by including TABLAT in the RS232 reception ISR function

#pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata")
void vSerial2RxISR( void )
{
static portCHAR cChar,cWoken;

Now communication is flawless.
I guess I have to do the same thing with the other ISR functions.

Greetings

Paul van der Hulst

Hi,

I finally tracked down the problem.

As you remember I had problems with serial i/o

The situation: An ISR sends data to the queue (at 38000 baud)
The communication task reads the data from the queue through the intermediate function cSerial2GetChar.

Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE
without having been blocked.

I checked this by keeping track of the TickCount before entering the function and after returning.

The problem lies in the generated assembly code. I included the C code and the assembly section below:

-----------------------------------------------------------------------------------
if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) )
{
return ( portCHAR ) pdTRUE;
}


14291 6FA4 ECEE CALL cQueueReceive, 0
14292 6FA6 F023 NOP
14293 6FA8 6EF5 MOVWF TABLAT, ACCESS
14294 6FAA 0E06 MOVLW 0x6
14295 6FAC 5CE1 SUBWF FSR1L, W, ACCESS
14296 6FAE E202 BC 0x6fb4
14297 6FB0 6AE1 CLRF FSR1L, ACCESS
14298 6FB2 52E5 MOVF POSTDEC1, F, ACCESS
14299 6FB4 6EE1 MOVWF FSR1L, ACCESS
14300 6FB6 50F5 MOVF TABLAT, W, ACCESS
14301 6FB8 0900 IORLW 0
14302 6FBA E003 BZ 0x6fc2

-----------------------------------------------------------------------------------
After returning from the call, WREG is copied to TABLAT (line 14293)
In line 14300 TABLAT is copied back to WREG and the result is tested (the 'if' condition).

Apparently TABLAT is changed between returning from the call and the evaluation at line 14301.
This can be either a task-switch or an ISR. In this case it is an ISR, ISRs don't save TABLAT by default.

I'm still trying to find out what the intermediate code is for,
I have not been able to reproduce this.

It probably has something to do with the function parameters being pointers or something like that.
Functions which only return a 'char' value and have no or only one char parameter produce the code:

call functionname,0
nop
iorlw 1
bz 0xxxxx <-- destination address


I fixed this by including TABLAT in the RS232 reception ISR function

#pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata")
void vSerial2RxISR( void )
{
static portCHAR cChar,cWoken;

Now communication is flawless.
I guess I have to do the same thing with the other ISR functions.

Greetings

Paul van der Hulst

Well, sorry about double posting, remove some if possible...
I posted the message but is wouldn't display.

Again, sorry

Paul van der Hulst

I am developing a project for LPC2103. For this I took ARM7_LPC2106_GCC demo as base. Till now I am using heap2.c and my code is working fine.
Now, I want to use heap3.c. For this , in Linker script I setup the heap after bss section like,

.heap :
{
__heap_start__ = .;
*(.heap)
. += 2k;
__heap_end__ = .;
} >ram

But my code is not is not working. Do I have to do something more than this?