I've noticed that entities tend to disappear when they're in a chunk with the Exotic Gardens plants. They're all still there, just can't be seen. Items, mobs (Both passive and hostile), and even players, disappear when in the same chunk as an Exotic Garden plant.
From the looks of it, you brought Lapis plants into your Enderman farm, putting them right next to the cows. If you were to move the plants (Or the cows...) so that they were in different chunks, then you shouldn't have this problem any longer. You can see chunk boundaries with "F3+G", in case you didn't know.
Note: While the fruit plants haven't really been tested as thoroughly as the magical plants, I'd assume they'd be pretty similar.