diff options
| author | Andrea Arcangeli <aarcange@redhat.com> | 2011-01-13 15:46:42 -0800 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2011-01-13 17:32:40 -0800 |
| commit | 64cc6ae001d70bc59e5f854e6b5678f59110df16 (patch) | |
| tree | 929358dabd522330ea145755ab0d0d7d27ac85b3 /arch/x86/mm/gup.c | |
| parent | db3eb96f4e6281b84dd33c8980dacc27f2efe177 (diff) | |
thp: bail out gup_fast on splitting pmd
Force gup_fast to take the slow path and block if the pmd is splitting,
not only if it's none.
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Acked-by: Rik van Riel <riel@redhat.com>
Acked-by: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'arch/x86/mm/gup.c')
| -rw-r--r-- | arch/x86/mm/gup.c | 13 |
1 files changed, 12 insertions, 1 deletions
diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c index 06f56fcf9a77..269aa53932e0 100644 --- a/arch/x86/mm/gup.c +++ b/arch/x86/mm/gup.c @@ -160,7 +160,18 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end, pmd_t pmd = *pmdp; next = pmd_addr_end(addr, end); - if (pmd_none(pmd)) + /* + * The pmd_trans_splitting() check below explains why + * pmdp_splitting_flush has to flush the tlb, to stop + * this gup-fast code from running while we set the + * splitting bit in the pmd. Returning zero will take + * the slow path that will call wait_split_huge_page() + * if the pmd is still in splitting state. gup-fast + * can't because it has irq disabled and + * wait_split_huge_page() would never return as the + * tlb flush IPI wouldn't run. + */ + if (pmd_none(pmd) || pmd_trans_splitting(pmd)) return 0; if (unlikely(pmd_large(pmd))) { if (!gup_huge_pmd(pmd, addr, next, write, pages, nr)) |