The aioli will appear quite tight when all the oil is incorporated, but it will loosen to a pourable consistency once the buttermilk and shallot are added.
Mix shallot and lemon juice in a small bowl. Whisk egg yolk, garlic, kosher salt, cayenne, and 2 teaspoons water in a medium bowl. Whisking constantly, gradually drizzle in vegetable oil, drop by drop at first, until aioli is thickened and smooth. Gently mix in buttermilk, crème fraîche, and shallot mixture; season with kosher salt and black pepper.
Prepare grill for medium-high heat.
Toss half of snap peas with 1 tablespoon olive oil in a medium bowl; season with kosher salt and black pepper. Set a wire rack or grill basket on grill grate and grill peas on rack, turning occasionally, until lightly charred, about 4 minutes; return to bowl.
Cut raw peas in half crosswise on a diagonal and toss with grilled peas along with remaining 1 tablespoon olive oil; season with black pepper. Spoon some aioli on a platter and top with snap peas, chiles, and sea salt. (Reserve remaining buttermilk aioli for another salad, or use it as a dip.)
Do ahead: Buttermilk aioli can be made 3 days ahead. Cover and chill.
