kmalloc()

kmalloc() is defined as inline function in ${linux src}/include/linux/slab.h

static inline void *kmalloc(size_t size, unsigned int __nocast flags)
{
        if (__builtin_constant_p(size)) {
                int i = 0;
#define CACHE(x) \
                if (size <= x) \
                        goto found; \
                else \
                        i++;
#include "kmalloc_sizes.h"
#undef CACHE
                {
                        extern void __you_cannot_kmalloc_that_much(void);
                        __you_cannot_kmalloc_that_much();
                }
found:
                return kmem_cache_alloc((flags & GFP_DMA) ?
                        malloc_sizes[i].cs_dmacachep :
                        malloc_sizes[i].cs_cachep, flags);
        }
        return __kmalloc(size, flags);
}

__builtin_constant_p() macro is gcc builtin function.
Here is an extract from info of gcc.

`__builtin_constant_p (EXP)'
     You can use the builtin function `__builtin_constant_p' to
     determine if a value is known to be constant at compile-time and
     hence that GNU CC can perform constant-folding on expressions
     involving that value.  The argument of the function is the value
     to test.  The function returns the integer 1 if the argument is
     known to be a compile-time constant and 0 if it is not known to be
     a compile-time constant.  A return of 0 does not indicate that the
     value is *not* a constant, but merely that GNU CC cannot prove it
     is a constant with the specified value of the `-O' option.

You would typically use this function in an embedded application where memory was a critical resource. If you have some complex calculation, you may want it to be folded if it involves constants, but need to call a function if it does not.

When required size for memory allocation is the size of compile-time constant,
"if" block is executed.

Inside the block, CAECH() macro is defined and linux/kmalloc_sizes.h header
is included.
These code does check whether the value passed as argument is smaller or not.
And when it smaller than the size passed CACHE(), goto found: label.
Otherwize, increment local variable "i", which is the index for malloc_sizes[].

When the required size is under size that is defined in kmalloc_sizes.h,
kmem_cache_alloc() is called.

kmem_cache_alloc() is defined in ${linux src}/mm/slab.c

void *kmem_cache_alloc(kmem_cache_t *cachep, unsigned int __nocast flags)
{
        return __cache_alloc(cachep, flags);
}

This function is only wrapper function for __cache_alloc().
__cache_alloc() is also in slab.c

static inline void *__cache_alloc(kmem_cache_t *cachep, unsigned int __nocast fl
ags)
{
        unsigned long save_flags;
        void* objp;
        struct array_cache *ac;

cache_alloc_debugcheck_before(cachep, flags);

local_irq_save(save_flags); ac = ac_data(cachep); if (likely(ac->avail)) { STATS_INC_ALLOCHIT(cachep); ac->touched = 1; objp = ac_entry(ac)[--ac->avail]; } else { STATS_INC_ALLOCMISS(cachep); objp = cache_alloc_refill(cachep, flags); } local_irq_restore(save_flags); objp = cache_alloc_debugcheck_after(cachep, flags, objp, __builtin_retur n_address(0)); return objp; }

cache_alloc_debugcheck_before() is also in slab.c

cache_alloc_debugcheck_before(kmem_cache_t *cachep, unsigned int __nocast flags)
{
        might_sleep_if(flags & __GFP_WAIT);
#if DEBUG
        kmem_flagcheck(cachep, flags);
#endif
}

If DEBUG is not defined, it only calls might_sleep_if().
About might_sleep_if(), look at __alloc_pages().

kmem_cache_t is typedef to struct kmem_cache_s.
struct kmem_cache_s is defined in ${linux src}/mm/slab.c, a little huge struct.
This struct is used to manage page caches.
The first element of this structure is struct array_cache.

struct array_cache (described later) keeps information of cache.

ac_data() and ac_entry() are in slab.c

static inline struct array_cache *ac_data(kmem_cache_t *cachep)
{
        return cachep->array[smp_processor_id()];
}

static inline void **ac_entry(struct array_cache *ac) { return (void**)(ac+1); }

If array_cache named "ac" is available(ac->avail > 0), it get page from cache.

                objp = ac_entry(ac)[--ac->avail];

In these inline functions, there are some confusing things.
ac_entry() returns (void**)(ac + 1), in which ac points struct array_cache.
(ac + 1) points just after the array_cache specified by ac.

struct array_cache is defined in ${linux src}/mm/slab.c

struct array_cache {
        unsigned int avail;
        unsigned int limit;
        unsigned int batchcount;
        unsigned int touched;
};

When this structure is initialized, process to do that is as following code:

static struct array_cache *alloc_arraycache(int cpu, int entries,
                                                int batchcount)
{
        int memsize = sizeof(void*)*entries+sizeof(struct array_cache);
        struct array_cache *nc = NULL;

if (cpu == -1) nc = kmalloc(memsize, GFP_KERNEL); else nc = kmalloc_node(memsize, GFP_KERNEL, cpu_to_node(cpu));

if (nc) { nc->avail = 0; nc->limit = entries; nc->batchcount = batchcount; nc->touched = 0; } return nc; }

So, real array of cache is following the struct array_cache, which results
in that (ac + 1) points the beginning of array of pointer for objects.

ac_entry() returns pointer of array of pointers of object (void **)
that is cached with this strcut array_cache.

The following [--ac_avail] returns the last of array and adjust the
number of remaining object.

When DEBUG is not defined, cache_alloc_debugcheck_after() is defined as
following in slab.c

#define cache_alloc_debugcheck_after(a,b,objp,d) (objp)

It does do nothing.

__kmem_cache_alloc() is called with argument malloc_sizes[].
malloc_sizes[] are defind in slab.c as an array of struct cashe_sizes.

strut cache_sizes is defind in ${linux src}/include/linux/slab.h

struct cache_sizes {
        size_t           cs_size;
        kmem_cache_t    *cs_cachep;
        kmem_cache_t    *cs_dmacachep;
};

After all, when the size is compile time size, the system tries to get pages
from these caches.

Otherwize, it leaves the job to __kmalloc().

__kmalloc()

__kmalloc() is defined in ${linux src}/mm/slab.c

void *__kmalloc(size_t size, unsigned int __nocast flags)
{
        kmem_cache_t *cachep;

cachep = __find_general_cachep(size, flags); if (unlikely(cachep == NULL)) return NULL; return __cache_alloc(cachep, flags); }

This function at first calls __find_general_cachep().
__find_general_cachep() is also defined in slab.c

static inline kmem_cache_t *__find_general_cachep(size_t size,
                                                unsigned int __nocast gfpflags)
{
        struct cache_sizes *csizep = malloc_sizes;

#if DEBUG BUG_ON(csizep->cs_cachep == NULL); #endif while (size > csizep->cs_size) csizep++;

if (unlikely(gfpflags & GFP_DMA)) return csizep->cs_dmacachep; return csizep->cs_cachep;

This function also get required pages from malloc_sizes[] cachees, normal or dma.
While size of pages that is cached in malloc_sizes[] array is smaller than
size required, pointer is incremented (csizep++), wihch means next large size
should be checked.

When size in malloc_sizes[some index] is enough to include size erquired,
it cals __cache_alloc() with csizep->cs_{dma}cachep and
it gets pages from csizep->cs_cachep or csizep->cs_dmacachep
according to FGP_DMA bit on gfpflags.

top
inserted by FC2 system