The pressures against your thumb and inside the barrel may be but are not necessarily the same because the end of the plunger that is against your thumb is not necessarily the same size/diameter as the syringe barrel.

Pressure is calculated by P = F/A where P is pressure, F is force, and A is the area over which the force is applied. If we assume a fixed (i.e., stationary) position of the plunger, then Newton's laws tell us that the forces (not pressures) on the plunger are balanced, meaning they have the same magnitude but are pointing in opposite directions. Therefore, in calculating the pressure for either the thumb-end or the barrel side of the plunger, the same F is used. But if the end of the plunger against your thumb is a different size than the end that is in the barrel of the syringe, then the areas will be different, and therefore the pressures will be different.

To determine the ratio between the two, rewrite the force balance in terms of the pressures and areas:
F_thumb = P_thumb * A_thumb = P_barrel * A_barrel = F_barrel.

Rearrange to find one pressure in terms of the other:
P_thumb = P_barrel * A_barrel / A_thumb.

Conceptually this is similar to a nail and the different forces at the head struck by the hammer and the point that is driven into wood (point has much smaller area and therefore much larger pressure even though the force is the same as at the head).

A related question concerns the force used to block the end of the nozzle. According to Pascal's Law/Principle, pressure (not force) is transmitted at the same magnitude throughout a fluid. This means that the fluid pressure at the end of the nozzle (not to mention against the side walls of the barrel) is the same as from the fluid against the end of the plunger. (Note that pressure does not have a direction, but the force associated with the pressure is directed perpendicular to the surface against which the pressure is exerted.) Therefore, the problem here requires setting the pressures equal:
P_nozzle = F_nozzle / A_nozzle = F_plunger / A_plunger = P_plunger.

However, similar to the original question, the areas are probably different - presumably the nozzle is much smaller than the barrel. So the force at the end of the nozzle F_nozzle = F_plunger * A_nozzle / A_plunger is smaller than the force at the plunger.

This is a working principle of hydraulic systems where pistons of different sizes are connected through a fluid and a small force on a small piston can be used to create a large force on a larger piston. The tradeoff is in the amount of displacement - the length of a cylinder of fluid ejected from the small nozzle (assuming it doesn't spread outward and stays at the diameter of the nozzle) would be larger than the distance the plunger is depressed.

Going the other way, using a small end of a hydraulic system to drive a large end means that the output force is larger but the distance over which it acts is smaller.