Isaiah 53 should be read in context with its surrounding verses. The surrounding verses talk about Israel, not Jesus as the context is about the suffering of Israel as a result of the Babylonian exile. Isaiah 52 speaks about Israel and there is no indication that the subject changes going into Isaiah 53. 

Isaiah 52:13 Behold, my servant shall prosper, he shall be exalted and extolled, and be very high.​14 As many were astonished at you; saying His appearance is too marred to be that of a man, and his forms to be that of the sons of men;​ 15 So shall he startle many nations; kings shall shut their mouths at him; for that which had not been told them shall they see; and that which they had not heard shall they perceive. 

Isaiah 53:​1 Who has believed our report? and to whom is the arm of the LORD revealed?​ 2 For he grew up before him as a tender plant, and as a root out of a dry ground; he had no form nor comeliness that we should look at him, there was no countenance that we should desire him.​3 He was despised and rejected by men; a man of sorrows, and acquainted with sickness; and we hid as it were our faces from him; he was despised, and we esteemed him not.​4 Surely he has borne our sicknesses, and carried our sorrows; yet we esteemed him stricken, struck by God, and afflicted.​5 But he was wounded because of our transgressions, he was bruised because of our iniquities; his sufferings were that we might have peace; and by his injury we are healed.​6 All we like sheep have gone astray; we have turned every one to his own way; and the the LORD has laid on him the iniquity of us all.​7 He was oppressed, but he humbled himself and opened not his mouth; he was brought like a lamb to the slaughter, and like a sheep, that is dumb before its shearers, he did not open his mouth.​ 8 By oppression and false judgment was he taken away; and of his generation who considered? For he was cut off from the land of the living, stricken for the transgression of my people.​9 And they made his grave among the wicked, and his tomb among the rich; although he had done no violence, and there was no deceit in his mouth. ​10 Yet it pleased the LORD to crush him by sickness; if his soul shall consider it a reward for guilt, he shall see his seed, he shall prolong his days, and the purpose of the LORD shall prosper in his hand.​11 He shall see the labor of his soul, and shall be satisfied; by his knowledge did my servant justify the righteous One to the many, and did bear their iniquities.​12 Therefore I will give him a portion with the great, and he shall divide the plunder with the strong; because he has poured out his soul to death; and he was numbered with the transgressors; and he bore the sin of many, and made intercession for the transgressors.

Isaiah 52:13-15 is God speaking, predicting the exaltation and triumph of Israel and the astonishment of the Gentiles at Israel’s triumph. 

From Isaiah 53:1 onwards it is the astonished Gentile nations speaking about Israel, God’s servant. 

In verse 1 the Gentiles admit that the arm of the LORD has been revealed to Israel. Other verses in the Tanakh support this, one of them is Exodus 6​:6 “Therefore say to the people of Israel, I am the LORD and I will bring you out from under the burdens of the Egyptians, and I will rid you from their slavery, and I will redeem you with a outstretched arm, and with great judgments; 7 And I will take you to me for a people, and I will be to you a God; and you shall know that I am the LORD your God, who brings you out from under the burdens of the Egyptians.” 

In verses 3 and 4 the Gentiles admit that the servant suffered because of their iniquities. Christian versions of the Bible like to translate the “from our iniquities” מפשענו in verse 5 as “for our iniquities” which sounds more like the sacrificial death of Jesus. 

Isaiah 53:8 says the servant was cut off from the land of the living. Christians insist this must refer to the death of Jesus as a nation cannot die, but this refers to the Jews being expelled from Judah during the Babylonian exile. When the Jews were expelled, the nation of Israel was cut off. This imagery is used in Ezekiel, when the nation of Israel is in exile, they are symbolically dead:

Ezekiel 37:​9 And he said to me, Prophesy to the breath, prophesy, son of man, and say to the breath, Thus says the LORD God: Come from the four winds, O breath, and breathe upon these slain, that they may live.​10 And I prophesied as he commanded me, and the breath came into them, and they lived, and stood up upon their feet, a very large army.​11 Then he said to me, son of man, these bones are the whole house of Israel; behold, they say, Our bones are dried, and our hope is lost; we are clean cut off.​ 12 Therefore prophesy and say to them, Thus says the Lord God: Behold, O my people, I will open your graves, and cause you to come up out of your graves, and bring you into the land of Israel.​13 And you shall know that I am the Lord, when I have opened your graves, O my people, and brought you up from your graves,​ 14 And I shall put my spirit in you, and you shall live, and I shall place you in your own land; then shall you know that I, the LORD, have spoken it, and performed it, says the LORD.

Isaiah 53:9 says the servant had done no violence and spoken no falsehood. The Jews had done nothing to cause the persecution they suffered. The Babylonian exile was the result of imperialist expansion and later persecution in Europe was caused by Christians who saw the Jews as Christ killers who rejected their messiah. 

Isaiah 53:10 says that the servant shall see seed (have children) and that his days will be prolonged. This does not fit Jesus who died at about age 30 and had no children. Verse 10 also says the servant will see it is a reward for guilt. This fits the nation of Israel who has sinned against God many times but not Jesus who was supposedly sinless. 

Israel is the servant

The nation of Israel is named as God’s servant many times in the Tanakh, especially in the book of Isaiah:

lsaiah 41:​8 But you, Israel, are my servant, Jacob whom I have chosen, the seed of Abraham my friend.​9 You whom I have taken from the ends of the earth, and called you from its farthest corners, and said to you, You are my servant; I have chosen you, and not cast you away.

Isaiah 44:​1 Yet now hear, O Jacob my servant; and Israel, whom I have chosen;​ 2 Thus says the LORD who made you, and formed you from the womb, who will help you; Fear not, O Jacob, my servant; and you, Jeshurun, whom I have chosen.

Isaiah 45:4 For Jacob my servant’s sake, and Israel my elect, I have called you by your name; I have surnamed you, though you have not known me.

Isaiah 49:​1 Listen, O islands, to me; and listen, you people, from far; the LORD has called me from the womb; from the bowels of my mother has he made mention of my name.​2 And he has made my mouth like a sharp sword; in the shadow of his hand he hid me, and made me a polished shaft; in his quiver he hid me.​3 And said to me, You are my servant, O Israel, in whom I will be glorified.

Jeremiah 30:​10 Therefore do not fear, O my servant Jacob, says YHWH; nor be dismayed, O Israel; for, behold, I will save you from afar, and your seed from the land of their captivity; and Jacob shall return, and shall be quiet and at ease, and none shall make him afraid.

All this supports the Jewish and the Yahwist viewpoint that Jesus is not the subject in Isaiah 53. You could argue that this supports Jesus allegorically but not as prophecy like the missionaries argue.