most of all Central Park in the Dark and The Unanswered Question, plus some of the piano music. The Concord Sonata is wonderful to hear live, but he often loses his bearings in the longer pieces. Bernstein or Michael Tilson Thomas are the best conductors for his music.
I don't remember if he had much to say about Indiana. A quick check reveals all: Peru's Cole Porter is overrated but Richmond's Ned Rorem is at least mentioned.
Wikipedia, complete with unsourced assertion, on Central Park in the Dark:
In 1906 Ives would compose what some have argued was the first radical musical work  of the twentieth century, "Central Park in the Dark". The piece simulates an evening comparing sounds from nearby nightclubs in Manhattan (playing the popular music of the day, ragtime, quoting "Hello My Baby") with the mysterious dark and misty qualities of the Central Park woods (played by the strings).
And a previous aworks post about New Jersey composers, including Milton B.