Although some travel guides warn about harassment. In my opinion this is not a problem in Port Antonio. You may be harassed in Montego Bay or in Ocho Rios, but it is very unlikely that this happens here. If anyone annoys you, just say "no, thank you", smile and leave. If you do not make eye contact with anyone "suspicious", it is very unlikely that something like this happens.
If you need help, you can ask anyone you meet. 99% of Jamaicans are nice and friendly people. If you have a problem with anyone who belongs to the rest 1%, ask for help.