I have very little experience with anything but single-color cloth backdrops. In general, I like simple backdrops that are neutral in color because I don’t have to worry about it pulling attention to itself.
Backdrops go hand in hand with studio lighting. I’ve tried using backdrops without studio lighting and have run into nothing but problems and results that I do not like. I’m in the process of figuring out a good, portable, 3-point light setup so I can shoot indoors using something other than a natural background and actually get results that I like. So, just a word of warning. The decision about what backdrop to use usually comes after all the lighting decisions have been made.
So, since my observation is that backdrops require studio lighting, I find that backdrops that try to simulate a natural setting don’t really fit. They look “fake” and thus “cheap” somehow, regardless of how expensive they actually may have been in price. They are very hard to make look “right”. I think natural, outdoor backgrounds look in place when shot under natural, outdoor lighting.
You didn’t mention textured or patterned backdrops, but I would say that they probably tend to unnecessarily date photos just like fads or trends in clothing.