Can a Man forgive sins?

Every Sunday it is common practice at the very beginning of the Divine Service for the congregation to confess our sins. (We also have private Confession and Absolution, where a person privately confesses sins that are troubling them directly to the pastor. Whether you're familiar with the former, latter, or neither, the following applies to all.) There is usually a few seconds of silence to allow people to call to mind those sins that might particularly trouble them. Then, together, we say...
I, a poor, miserable sinner, confess unto You all my sins and iniquities with which I have ever offended You and justly deserved Your temporal and eternal punishment. But I am heartily sorry for them and sincerely repent of them, and I pray You of Your boundless mercy and for the sake of the holy, innocent, bitter sufferings and death of Your beloved Son, Jesus Christ, to be gracious and merciful to me, a poor, sinful being. 
The pastor as a sinner confesses these words with everyone else. When these words are spoken the pastor then turns around and faces the congregation and says:
Upon this your confession, I, by virtue
of my office, as a called and ordained
servant of the Word, announce the
grace of God unto all of you, and in
the stead and by the command of my
Lord Jesus Christ I forgive you all
your sins in the name of the Father and
of the T Son and of the Holy Spirit. 
Now, I mention this practice because in the Lutheran church, beginning a service like this is not common among the various other churches and is very unique to those of the Lutheran confession. Due to this, it is not uncommon to hear occasional scandalized reactions from visitors attending a Lutheran church for the first time upon hearing the words of the pastor. Statements such as, "Who does he think he is? Only God can forgive sins." While it hasn't happened to me personally, I have pastor friends who have experienced visitors getting up and walking out upon hearing these words. So, what should we make of it? Can man forgive sins? Isn't it true that only God forgives sins?
Now, as a pastor, when asked, I usually have a bit of fun and respond, "AMEN! Preach it, Brother! Only God can forgive sins?" This often elicits blank, bewildered stares because they are expecting me to defend my holy authority to forgive sins.
Then, with seriousness, I point them back to the words I spoke. I didn't say, "By virtue of how totally awesome, amazing, and holy person I am, I forgive all your sins because I'm so dazzlingly fabulous." No, the forgiveness I offer is not my own. I am not forgiving anyone's sins. Jesus is the one forgiving sins.
Now, it might be challenging for us to fully grasp this concept because the Church in the West, in our age, has suffered from an erosion of the Office of the Holy Ministry. This erosion may have occurred due to the desire among Protestants to avoid the excesses of the Roman Catholic Church. Rome teaches that in the sacrament of ordination, the man being ordained receives what is called an indelible mark. Something changes within him (similar to wine turning into blood), so that the priest can forgive sins, administer baptism, and perform other sacraments. One can rightly recognize the problems with this and, in an effort to correct the abuse of Rome, the pastor becomes just a regular guy whom the church pays to think more deeply about things and deliver his thoughts and ponderings at a weekly meeting every Sunday. You may choose to agree with him or not. The former turns the pastor into a superhuman, a god among men, while the latter renders him a neutered, defanged, and declawed cat – soft and cuddly but otherwise impotent.
 
It's always easy to fall into extremes, but much harder to plot a course between said extremes. I think this is where the Lutheran doctrine of the Office of Holy Ministry plots a biblical course between these extremes.
Imagine if I got very drunk and proceeded to swerve in and out of traffic at 120 km/h in a 50 km/h zone. I am then pulled over by the police. The police officer who pulls me over happens to be my high school buddy Jon, with whom I used to play video games in his mom's living room after school every day. Jon proceeds to tell me to get out of the car, handcuffs me, and arrests me as a danger to the public. In response, I scream, "Who do you think you are, Jon? You always thought too highly of yourself. You have no authority to do this, Jon."
Hopefully, we can recognize the problem with what I said to Jon. My argument is that Jon is just a regular guy, my buddy and pal, on an ego trip, drunk on power. I would be right in acknowledging that he is just a regular guy. However, he is a regular guy who is an officer of the law. And as such, he arrests me not from his own power or authority as an individual, but as an officer of the law who is tasked with enforcing the law on behalf of the authority of the state.
In a similar manner, this is what happens when the pastor pronounces absolution. The pastor has no superpowers. It is not he who is forgiving sins, but rather it is Christ speaking through one of his officers. The pastor, as a poor miserable sinner just like you, holds the office and speaks forgiveness as Christ commanded to repentant sinners. Now, you might ask if this is just a nice tradition. The answer to that is found in John 20:19-23, a passage that often surprises people who haven't given much thought to it. Yet, it stands as the very foundation of this teaching. The passage reads as follows:
"On the evening of that day, the first day of the week, the doors being locked where the disciples were for fear of the Jews, Jesus came and stood among them and said to them, 'Peace be with you.' When he had said this, he showed them his hands and his side. Then the disciples were glad when they saw the Lord. Jesus said to them again, 'Peace be with you. As the Father has sent me, even so I am sending you.' And when he had said this, he breathed on them and said to them, 'Receive the Holy Spirit. If you forgive the sins of any, they are forgiven them; if you withhold forgiveness from any, it is withheld.'"
 
Here we see Jesus sending out the Apostles, the first pastors and holders of the Office of Holy Ministry, to forgive sins. This has been the ongoing role of pastors ever since.
The Augsburg Confession, the foundational document of the Lutheran Confession in the Book of Concord, states the following regarding confession:
"Our people are taught that they should highly value Absolution as being God's voice and pronounced by God's command." (Augsburg Confession XXV:3)
Now, it's important to understand what I'm not saying here. I'm not saying that receiving the forgiveness of Christ from the pastor is required. For example, if you live in a remote part of the country where no pastor is available, it doesn't mean you are cut off from the forgiveness of sins or salvation. One can confess their sins privately to God. See, this is how the gifts of God, such as the Lord's Supper, Baptism, and in this case, Absolution, can be perverted and misconstrued. It happens when we turn them into commands or laws that must be observed or else. Absolution is not a command or a law; it is a pure gift. In other words, it is pure Gospel. The point is not to have you confess your sins and receive forgiveness from Christ through the mouth of the pastor, or else you'll go to hell. The point is to provide assurance and comfort to those who are burdened by their sin.
If someone sins, they certainly can pray for forgiveness, and it is truly theirs. However, this places things in the dark realm of subjectivity. And this is the realm in which Satan loves to keep Christians. He will whisper to the Christian, "Did God really forgive you? Were you really sincere enough? Is your faith strong enough? Was God even listening?" Satan will throw all kinds of ideas into our heads to torment us, and he particularly loves it when we remain in that realm of subjectivity because that's when he can do the most damage. Certain sins begin to fester and gnaw at you. And here comes the great gift of the comfort and assurance of Holy Absolution. It forces us out of the realm of subjectivity into the realm of the objective, which is truly the foot of the cross. In Holy Absolution, our sin is laid before Jesus, and Jesus speaks. It may sound like the pastor, but it is Jesus Himself speaking. He takes a laser to the sins that are tormenting and troubling you, and with laser-like precision, He forgives you. Jesus gives you the assurance and comfort of the Gospel. So, when Satan tells you all those lies, you can answer with objective certainty, "Shut up, Satan. Jesus has spoken absolution over that sin. Jesus does not lie."
So, can a man forgive sins? The answer is a firm no. Only God forgives sins. However, when we receive absolution from the pastor, it is not the pastor who is forgiving sins, but Jesus Himself. It is Jesus who forgives our sins, even when He uses a poor, miserable sinner to do it.