Tory Miller draws inspiration from the Madison farmers’ market at his restaurant, L’Etoile.
Madison, Wisconsin, is a bastion of Midwestern cuisine. The state’s capital, located about an hour west of Milwaukee and two hours northwest of Chicago (depending on traffic), is known for its cheese curds, provincial supper clubs, and the brandy old fashioned (the distiller Korbel claims Wisconsin is responsible for half of all their brandy sales). But it’s also home to the largest producer-only farmers’ market in the nation, where chefs like Tory Miller source Highland beef, tomatoes, cheese, and many of the items that’ll end up on their menus.
Miller is the coproprietor and executive chef of L’Etoile, a James Beard Award–winning fine dining jewel, and Graze, a gastropub established in 2010 with a focus on comfort foods like burgers and bibimbap. The two neighboring restaurants look out on Madison’s Capitol Square, home to the Dane County Farmers’ Market in warmer months.
Originally from Racine, Wisconsin, Miller left his hometown to attend the French Culinary Institute. He worked in New York restaurants like Judson Grill and Eleven Madison Park in the early 2000s before making what was supposed to be a pit stop in Madison. He landed at L’Etoile in 2003 under chef Odessa Piper, who led a near-revolutionary commitment to sourcing from local purveyors since opening the restaurant in 1976. After building menus from what was available at the market, Miller decided to stay.
I called up Miller to talk about his start in New York City and how being an adoptee has informed his approach to food and sourcing, but the conversation always came back to the Dane County Farmers’ Market. It has flourished and grown in tandem with L’Etoile, making Madison’s food scene more bountiful and diverse than ever.
One of the reasons you decided to stay in Madison was the Dane County Farmers’ Market, which both your restaurants face directly. Can you describe what the market is like?
The Dane County Farmers’ Market is producer-only. You really get the character of the food and the farmers themselves putting that care into their stands.
I know this is true everywhere, but Wisconsin has a certain sense of hometown pride. When you go to the market, everyone is wearing their Packers, Badgers, Brewers, and Bucks gear—it’s all about being from here and supporting that. I think that sense of pride carries through the stalls, and for me, that’s always been the draw of the market.
Getting to go out and talk to the farmers, seeing ingredients I didn’t expect—or even seeing seasonal produce I do expect, and the sense of pride farmers convey if they had a good year. It just reads so clearly and makes me want to support them, to go back to my restaurant and put it on the menu.
Was it always your plan to come back to Wisconsin?
No. I knew I wanted to leave New York. Every day I would board the train in Queens to work in either Brooklyn or Manhattan. Going to work, cooking for a certain type of clientele, and then going back home—I just didn’t like it.
So I moved to Madison, but I didn’t expect to stay. I had other places on my list, maybe out west or down south. My first job in Madison was at L’Etoile for Chef Odessa [Piper], and she introduced me to the market in 2003. I was like, “This farmers’ market is incredible,” but I only got to see the very end of the season. So I said, “I’ll stay through spring,” to see the market next season. I stayed throughout 2004, and I was like, “This is the place.” I bought L’Etoile from Chef Odessa the very next year.
What was the moment you realized the market was so special to you?
The telling moment for me was when I stopped at this beef farm. They’re not there anymore, but they were our beef purveyor for years and years after this moment. They were raising these Scottish Highland cattle, and they gave me these frozen beef shanks. I immediately returned to the kitchen and cooked them, and they were great. When I went back to the market, they said, “Oh, you should come out sometime. We’re only 25 minutes away, and you can see the animals.”
And I did. We went out after, got some gin and tonics, and we sat and talked about beef—what I wanted to do, what I wanted the beef to taste like. Like finishing pigs on acorns in Spain, we have farms that finish pigs with apples, hazelnuts, or hickory nuts grown here. They said, “Well, we can help you with that.” We adjusted the aging, so the beef would be aged for 28 days before coming in-house, and I even went to the butcher to tell them exactly how we wanted it cut. The trippy thing is that this was in 2003.
That instant relationship was the thing that has driven me since then. Not to get too deep, but I’m adopted, an orphaned person, so relationships are super important and interesting to me. I love creating those weird bonds, and that’s what keeps me here, I guess.
The restaurant felt like my own before it was.
What inspired you to buy L’Etoile?
Chef Odessa started this restaurant in 1976 in her early twenties. She was one of the first people doing this kind of thing—using local ingredients and engaging with farmers—the kind of chef whose level of greatness people will never understand. She went through all of the ’70s, ’80s, and ’90s and then won a James Beard Award in 2001.
I came at the right moment. I was only here for a year before Chef Odessa started hinting that if anyone would like to purchase the restaurant, she’d be open to talking about it. And we had a fantastic relationship from day one. When I came in, we hung out, and she wanted me to make her lunch. I went in the cooler and grabbed some trout, beets, and Swiss chard to make pan-seared rainbow trout with beets cooked in brown butter, braised rainbow chard, and horseradish oil. We just ate lunch. Then she said, “I don’t have a full-time job for you, but you can work part-time.”
I was like, “Yeah, okay, I’ll take it,” because the vibes felt right. We just had this relationship where we wanted to talk about food, but she was also very nurturing and guiding—I hesitate to say “mothering,” but that’s who she was. Chef Odessa was caring, and I no longer have a relationship with my adoptive family really. Before connecting to her as a chef, I connected with her on that level.
The restaurant felt like my own before it was. I got to test the waters and learn the market for a year and a half before we started talking about loans and figuring out if we could do this. And the only way that it happened was that Chef Odessa was willing to work with us on that. This was the right spot for me.
Why do you think the Midwest—and Madison specifically—gets overlooked as a culinary destination?
The Midwest is one of the richest agricultural regions in the nation. We have this German legacy, and people associate us with fried cheese and bratwurst. But to me, it’s the mixing of our cultures and our agriculture that makes Wisconsin so fun. Wherever great food grows, there will be great food to eat.
With Madison specifically, having access to such good food makes this a place where you can come and start something—and not just people coming from big cities like New York or Chicago or whatever. When you talk to chefs like Sean Pharr at Mint Mark, Jamie Hoang at Ahan, Itaru Nagano at Fairchild, Daniel Bonanno at A Pig in a Fur Coat, or Daniel Fox at Heritage Tavern—these chefs have very small restaurants, and they’re cooking every day.
Madison is a small town; it’s a college town and a government town, and we don’t necessarily get the hype that cities like Milwaukee and Chicago get. I feel like Madison is on the cusp of being there, but it’s funny to me to talk like someday we’ll be a real restaurant town—because we already are.