While spending a pleasant afternoon in the backyard garden, the kids and I discovered a few of these "caterpillars" on a chrysanthemum plant. They are not caterpillars, but sawfly larva. The most obvious visual difference between caterpillars (which remember are butterfly and moth larva) and sawfly larva are the number of prolegs. Caterpillars have five or fewer, sawflies have six or more.
Sawflies are in the Hymenoptera, the bees, wasps, and ants. The adults most resemble wasps. Not all the larva resemble caterpillars, some species have larva with a definite grub-like look.