Carefully note the wording the Lord uses here - "Your brother has something against you" (5:23). Clearly, you are the offending party! Either you have sinned against your brother in some way or he believes that you have. You have the responsibility to go to him if you believe that he is angry with you about something. It doesn't matter if his reasoning is valid or if he is justified in being upset with you, what matters is that he is offended and both of you should work toward immediate reconciliation. Of course, he too has a responsibility to come to you if he is offended (Matt. 18:15).
Angry feelings must be attended to quickly since they can easily lead to sin (in words and deeds). Thus, one must lay aside his anger the same day to prevent Satan from taking advantage of the opportunity (Eph. 4:26,27). This anger must be set aside in a genuine attempt at reconciliation. Truly, offering a gift to God is very important, but reconciliation takes precedence over it and all other duties and acts of worship. The greatest gift offered to God will not be accepted if it is made by one who is not willing to try to be reconciled to his brother. A proper relationship with our fellow man is more important than worship itself. One cannot love God and hold a grudge against his brother simultaneously (I John 4:20,21). This is the case because man is made in the image of God (Gen. 1:26,27), and one cannot hate someone made in God's image and still love God.
Matthew 5:25,26 reads - "Agree with your adversary quickly, while you are on the way with him, lest your adversary deliver you to the judge, the judge hand you over to the officer, and you be thrown into prison. Assuredly, I say to you, you will by no means get out of there till you have paid the last penny." The implication here is that you have done something wrong; your adversary has a case against you. In such circumstances, you should try to make friends quickly and defuse the situation by agreeing with him; you should try to be reconciled. Otherwise, you are likely to end up in prison! Jesus' meaning here is simple: If you wrong someone and don't do everything you can to reconcile with him, you will be judged and punished (perhaps by civil government, but definitely by God, ultimately; cf. Matt. 18:34).
Friends, we must love our fellow man! We must seek to "live peaceably with all men" to the best of our ability (Rom. 12:18). We must pursue reconciliation. If not, we will get what we deserve! Is there someone you need to pursue reconciliation with in your life? If so, what are you waiting for?