The books say that the fruit is dull green, but becomes purplish-red when mature. So if it gets reddish then I would see if I could easily detach it, but if not just leave a little longer. If it seems to be drying up, then the seeds certainly would be ready. Don't rush it, as seeds need to become ripe to germinate well.
Good luck with it.