Combine 2 frozen bananas, milk, Greek yogurt, honey and vanilla in a blender. Blend until smooth. Pour into popsicle molds. Freeze for 45 minutes to 1 hour.
Meanwhile towards the end of the 45 minutes, blend 2 frozen bananas, Greek yogurt, milk, cocoa powder. Blend until smooth.
Pour over banana layer in popsicle molds. Freeze for about 5 hours, until solid.
Run warm water over molds for about 30 seconds. Popsicle should release easily. If not, repeat.