FreeRTOS Support Archive
The FreeRTOS support forum is used to obtain active support directly from Real
Time Engineers Ltd. In return for using our top quality software and services for
free, we request you play fair and do your bit to help others too! Sign up
to receive notifications of new support topics then help where you can.
This is a read only archive of threads posted to the FreeRTOS support forum.
The archive is updated every week, so will not always contain the very latest posts.
Use these archive pages to search previous posts. Use the Live FreeRTOS Forum
link to reply to a post, or start a new support thread.
[FreeRTOS Home] [Live FreeRTOS Forum] [FAQ] [Archive Top] [March 2017 Threads] Precompile FAT partition into flashPosted by miclam on March 17, 2017 Hello,
I'm trying to use the FreeRTOS +TCP HTTP server which requires +FAT for which I want to create a partition within the upper part of the LPC1758's 512kb flash. Also I need to deliver default HTML files in the factory build. In the demos is an example which writes an HTML file from a static variable in flash to a ramdisk. In my situation, that would mean I'd copy a string from flash to flash. I'd like to avoid that redundance.
So my question is, if and how it is possible to precompile a fat partition including files so it can be flashed together with the firmware.
Best regards,
Michael
Precompile FAT partition into flashPosted by heinbali01 on March 18, 2017 Hi Michael, are you familiar with Linux tools?
In Linux you can create a so-called loop device. You can format that as a small FAT16 partition (with the smallest possible cluster size), and fill it with your HTTP-files. When that is done, you can flush and close the loop device and transform the contents of a binary file into one big declaration in C:
~~~
attribute((aligned( 4 )))
const uint8t ucHTTPDrive[ HTTPSIZE ] = {
/* here data. */
};
~~~
The alignment helps memcpy() , which works much faster with aligned pointers.
It would need a simple read-only +FAT driver that reads sectors from that image in flash. Please look at ff_ramdisk.c for an example of such a driver.
Precompile FAT partition into flashPosted by miclam on March 22, 2017 Hello Hein,
thanks for your advice, that would be an option. Even though I would need a solution for windows as well.
But reading more into FAT I found another problem. The minimum FAT16 size is 4.1MB. Does this account for the +FAT implementation as well? That would be definitly too big for my 512kb flash (where I intent to use 128-256 kb). Also the limited 32kb RAM is a problem, regarding all the buffers used for caching.
Might FAT12 be a solution? And is it possible to reduce the cluster size to 256byte, as this is the minimum write size the LPC1758 allows?
My initial motivation to implement +FAT was, that the +TCP http server requires it per default. How much work would it be to change it for e.g. the uip http-fs?
Best regards,
Michael
Precompile FAT partition into flashPosted by heinbali01 on March 22, 2017 Hi Michael,
that would be an option
Even though I would need a solution for windows as well.
As well? You can create the C source file under Linux and compile under Windows. The nice thing is that the Linux process can easily be automated with a script.
And is it possible to reduce the cluster size to 256byte,
as this is the minimum write size the LPC1758 allows?
I'm afraid that the 512-byte size is almost hard-coded into the library. The reason is that most media use 512-byte sectors.
My initial motivation to implement +FAT was, that the +TCP http server requires it per default.
How much work would it be to change it for e.g. the uip http-fs?
If your embedded web page is very simple, then you might just store a few HTTP files in flash and simulate the calls to ff_fopen() , ff_fread() , and ff_fclose() .
I have heard of uIP, but not "uip http-fs".
The minimum FAT16 size is 4.1MB.
Does this account for the +FAT implementation as well?
+FAT uses ffconfigMIN_CLUSTERS_FAT16 and ffconfigMIN_CLUSTERS_FAT32 to decrease these boundaries. By default, their values are 4086 and 65525 (clusters).
Might FAT12 be a solution?
+FAT also has FAT12 included, but to be honest, I don't know user's experiences with it. I rarely use it my self.
Also the limited 32kb RAM is a problem, regarding all the buffers used for caching.
A HTTP server, a +FAT driver, and a small RAM disk?
To put it positively, that's quite a challenge!
~~~
/* Some parameters to reduce the RAM foot-print of +TCP.
Settings are not yet tested. */
#define ipconfigNETWORK_MTU 552 /* To get an MSS of 512 bytes. */
#define ipconfigUSE_TCP_WIN 0 /* No TCP sliding windows. */
#define ipconfigNUM_NETWORK_BUFFER_DESCRIPTORS 3
#define ipconfigTCP_RX_BUFFER_LENGTH ( 2 * ipconfigTCP_MSS )
#define ipconfigTCP_TX_BUFFER_LENGTH ( 2 * ipconfigTCP_MSS )
~~~
Regards.
Precompile FAT partition into flashPosted by miclam on March 22, 2017 No RAM disk, but Fash disk. But I guess my best option will be to change the underlaying file system in the HTTP server. The http-fs I was talking about comes from Adam Dunkels httpd implementation: https://github.com/adamdunkels/uip/tree/master/apps/webserver
There the HTML files are stored as static variables directly in .data section in flash. Thus read-only, but nearly no overhead. As far as I have seen there are just 3 calls to fffopen, fffclose and ff_fread, I will change those and provide the sources. Do you see the possibility to officially abstract the filesystem in the HTTP server, so +TCP does not depend on +FAT anymore?
Thanks for the +TCP parameters, I'll try them out!
Regards, Michael
Precompile FAT partition into flashPosted by rtel on March 22, 2017 I've not read the whole thread, so maybe my comments are not
appropriate, but:
No RAM disk, but Fash disk. But I guess my best option will be to change
the underlaying file system in the HTTP server. The http-fs I was
talking about comes from Adam Dunkels httpd implementation:
https://github.com/adamdunkels/uip/tree/master/apps/webserver
If this is the uIP webserver I'm familiar with then the file system is
not really a file system, but an array of character arrays. That would
be very difficult to make writable unless all your files are the same
length, or you don't mind having lots of padding at the end of files so
each file can consume the same space.
Do you see the possibility to
officially abstract the filesystem in the HTTP server, so +TCP does not
depend on +FAT anymore?
I would be grateful if you could elaborate here. As you imply, +TCP
does not depend on +FAT, but the HTTP server is always going to depend
on a file system. It is provided with +FAT, and +FAT already has an
abstraction layer to give it a 'standard' (stdio) API option (and that
is the API that is documented) - albeit with ff_ appended to the
expected stdio function names.
Precompile FAT partition into flashPosted by miclam on March 23, 2017
If this is the uIP webserver I'm familiar with then the file system is
not really a file system, but an array of character arrays. That would
be very difficult to make writable unless all your files are the same
length, or you don't mind having lots of padding at the end of files so
each file can consume the same space.
Yes, you are right. But as HTTP just requires a read-only filesystem this is enough.
By abstraction I mean, if it is possible to further abstract the fffopen, fffclose and fffread methods, so that it is possible to replace them. Please see my attached modifications in FreeRTOSHTTPserver.c and FreeRTOSserver_private.h . Basically I just changed the mentioned methods and the file handle data type. My modifications are marked with an // ML: comment.
Also I changed #include "ff_stdio.h" and as in FreeRTOSserverprivate.h HTTP and FTP are mixed I commented out the +FAT specific definitions in struct xFTP_CLIENT.
Precompile FAT partition into flashPosted by miclam on March 23, 2017 Now that I could actually run my code, I have an updated version of the FreeRTOSHTTPserver.c and FreeRTOSserverprivate.h .
Precompile FAT partition into flashPosted by rtel on March 23, 2017 Forgive me for saying, but this seems backward.
FreeRTOSHTTPServer.c is a file that has to work with multiple file
systems. The normal way of doing that is to use some kind of thin port
layer, or abstraction, that is completed for each file system. Most
file systems have a similar stdio style interface, so this is a simple
task. FreeRTOSHTTPServer.c stays the same and you somehow externally
map the function calls made by FreeRTOSHTTPServer.c to the API used by
your file system.
If you instead updated FreeRTOSHTTPServer.c so it can be used with a
specific file system, and then later want to change file systems, or use
FreeRTOSHTTPServer.c in a different project that has a different file
system, then you will end up with two inconsistent versions of
FreeRTOSHTTPServer.c.
Are you asking us to update +FAT so it works with your updated
FreeRTOSHTTPServer.c?
Precompile FAT partition into flashPosted by miclam on March 23, 2017
Are you asking us to update +FAT so it works with your updated
FreeRTOSHTTPServer.c?
No, the other way around. I'd like to have HTTP without dependency on +FAT.
Currently FreeRTOSHTTPServer.c depends on +FAT by including ff_stdio.h and using the ff_* methods.
FreeRTOSHTTPServer.c is a file that has to work with multiple file
systems. The normal way of doing that is to use some kind of thin port
layer, or abstraction, that is completed for each file system. Most
file systems have a similar stdio style interface, so this is a simple
task. FreeRTOSHTTPServer.c stays the same and you somehow externally
map the function calls made by FreeRTOSHTTPServer.c to the API used by
your file system.
That is what I wish for, some kind of port abstraction maybe in portable/FileSystem/ with subdirectories like Plus-FAT or http-fs or something else, that one can include/exclude from build. If there is a common header FileSystem.h (in analogy to NetworkInterface.h) which defines TCPfopen, TCPfread, ... (or a more suitable name) and an abstracted file handle type, that header can be used in FreeRTOSHTTPServer.c .
Finally in the port files one can write wrapper methods like my "prvfread" or simply define TCPfopen to ff_fopen.
Forgive me for saying, but this seems backward.
My implementation was actually quick and dirty to prove that the http-fs will work (it does) and to show that HTTP does not need to depend on +FAT. I would be glad if a file system abstraction is wished by you and I'm looking for help on doing so.
Precompile FAT partition into flashPosted by rtel on March 24, 2017
Currently FreeRTOSHTTPServer.c depends on +FAT by including
|ffstdio.h| and using the |ff*| methods.
I'm not following this. You can provide your own implementation of
ffstdio.h and the ff functions - that is how abstraction works - you
provide a shim between the function being called and the function you
want to map it too. If you change the name of the functions then you
will need to provide implementations of the new function names. What
has been gained? Currently the API is 'standard' in that it is what
stdio expects, but with ff_ prefixed to the function names so they don't
clash with the standard C functions. Changing it will make it harder to
port to different file systems as in most cases the port is just a
define from one to the other.
Precompile FAT partition into flashPosted by miclam on March 28, 2017
You can provide your own implementation of
ffstdio.h and the ff functions
Somehow I wasn't aware that I could simply overwrite +FAT's names if I'm not using it. That's what I did now, thanks to your advice, and it works great. No need to mess with FreeRTOSHTTPserver.c .
I append my implementation of ffstdio.h and ffstdio.c relying on httpd-fs 'file system' for which routines exist to convert text files into c-arrays, which can be loaded into data section of flash. Works fast and great for me.
Thanks a lot for your help,
Michael
Precompile FAT partition into flashPosted by heinbali01 on March 28, 2017 Hi Michael, thanks for sharing these sources. It looks good.
It is indeed common practice to use a set of defines. There is a library WolfSSL that has a port for +TCP and +FAT.
~~~
elif defined(FREERTOS_TCP)
#define RECV_FUNCTION(a,b,c,d) FreeRTOS_recv((Socket_t)(a),(void*)(b), (size_t)(c), (BaseType_t)(d))
#define SEND_FUNCTION(a,b,c,d) FreeRTOS_send((Socket_t)(a),(void*)(b), (size_t)(c), (BaseType_t)(d))
#define XFILE FF_FILE *
#define XFOPEN ff_fopen
#define XFSEEK ff_fseek
#define XFTELL ff_ftell
#define XREWIND(F) ff_fseek(F, 0, FF_SEEK_SET)
#define XFREAD ff_fread
#define XFWRITE ff_fwrite
#define XFCLOSE ff_fclose
#define XSEEK_END FF_SEEK_END
#define XBADFILE NULL
elif defined(FREESCALE_MQX)
~~~
Another thing I wanted to mention: if you want to create an image of a small FAT16 volume: you can create a RAM-disk in the FreeRTOS/Windows simulator. Fill it with files, and have the application make a dump of the image.
But you probably won't need that anymore. Regards
Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.
|