ruby寫一個檔案內容相似性比較的代碼

標籤：ruby   檔案相似性   外部編碼   

1.相似度定義

我們定義，則，我們設，則,|C|=s,則相似度p=，p（0,1）

2.相似度檢測演算法設計

演算法設計：

定義4個字元為一個字串，將T1，T2分割成若干字串，若剩餘字元不足4個，則以空格補全。將分割後的T1T2計數，記下|T1|=n，|T2|=m，s=0；在T1中取出第一字串，檢測是否在T2中，若存在，則s+1,並刪除與被檢測字串相同的字串，迴圈到T2檢測，直到T2中不存在被檢測的字串，迴圈到T1，提出下一個被檢測字串，到T2中檢測；如此迴圈檢測，直到T1中的所有字串都被檢測或者T2中所有的字串都被刪除，停止，記下此時的s；將所得的s除以n和m中最大的那個數，所得的結果為T1，T2的相似度。先以T1為被檢測模板，檢測，然後再以T2為被檢測模板檢測，得出兩個相似度的數，取最小值。

用ruby實現如下：

def fill_str(str,i=4)  return str if str.size%i == 0  str<<" "*(4-str.size%i)enddef txt_cmp(f0,f1)  str_f0,str_f1 = fill_str(File.new(f0).read),fill_str(File.new(f1).read)  a0,a1 = str_f0.scan(/.{4}/m),str_f1.scan(/.{4}/m)  n,m,s = a0.size,a1.size,0  a0.each do |txt|    if a1.include?(txt)      size = a1.size      s+=size-a1.keep_if {|item| item!=txt}.size    end    break if a1.size == 0  end  s/[n,m].max.to_frescue =>e  puts "error : #{e.message}\n" << e.backtrace[0..2].join("\n")end(puts "you must cmp 2 txt file";exit) if ARGV.size != 2r = txt_cmp(f0=ARGV[0],f1=ARGV[1])puts "#{f0} and #{f1} semblance is #{r*100}%"

下面是4個檔案分別為1.txt 2.txt a.txt b.txt，內容如下：

1.txt

NFC East rival quarterbacks Tony Romo(notes) of the Dallas Cowboys and Eli Manning(notes) of the New York Giants now have something else in common ?? they‘ve used the same wedding planner to help them tie the knot. Todd Fiscus, the man with the plan, set up what he called "man food" at Dallas‘ Arlington Hall on Saturday, when Romo married former Miss Missouri Candace Crawford. "I have a lot of football players to feed," said Fiscus, who had pizza and short ribs on the menu.

However, Romo apparently put all the tunes together. "Tony picked out every song, and when it plays, and what the keynote things are," Fiscus said.

Sounds like a very orderly occasion, but there was one wild card ?? whether Cowboys owner Jerry Jones would be able to attend. With the continued lockout, owners and players are not supposed to have any contact away from the negotiating table. But Jones received special dispensation from the NFL to attend, just as the Green Bay Packers recently were informed that they will, in fact, receive their Super Bowl rings in a June 16 ceremony no matter what the labor situation is at that time. Jones was there along with virtually all of Romo‘s teammates.

It is unknown whether Jones and Romo actually discussed any labor issues at the wedding ?? we‘re guessing this was more of a "friendly", though Jones is one of the most powerful owners on the NFL‘s side of things and Romo‘s marquee value gives him a lot of play on the other side.

"I‘ve gotten special permission," Jones recently told ESPN‘s Ed Werder. "But more than anything, (I got the) right ticket from him and his fiance?? ?? Romo‘s wife-to-be. (It‘s) one of prettiest invitations I‘ve ever seen.

"So, yes, I will be there and (I‘m) proud for him. He‘s got the best end of this deal."

Romo, who had been linked romantically before with Jessica Simpson and Carrie Underwood, proposed to Crawford last December. Crawford‘s brother Chace is known for his role on the TV show "Gossip Girl‘ and has also been linked romantically with Underwood.

According to the new Mrs. Romo, the lockout may play a part in the couple‘s plans for a honeymoon; usually around this time of year, her husband would be participating in minicamps and other off-season workouts.

"This lockout has been quite a dent in the honeymoon idea," she told WFAA-TV. "We‘ll see. We haven‘t really gotten there yet. We‘re taking a day at a time with the lockout. We (are not) even sure if we‘re gonna get to go (on) one."

2.txt

Officially, Memorial Day, observed on the last Monday of May (this year it‘s May 30), honors the war dead. Unofficially, the day honors the start of summer. (More on that in a moment.)

The upcoming three-day weekend has prompted searches on Yahoo! for "when is memorial day," "what is memorial day," and "memorial day history." The day was originally known as "Decoration Day" because the day was dedicated to the Civil War dead, when mourners would decorate gravesites as a remembrance.

The holiday was first widely observed on May 30, 1868, when 5,000 people helped decorate the gravesites of 20,000 Union and Confederate soldiers buried at Arlington National Cemetery. (Some parts of the South still remember members of the Confederate Army with Confederate Memorial Day.)

After World War I, the observances were widened to honor the fallen from all American wars--and in 1971, Congress declared Memorial Day a national holiday.

Towns across the country now honor military personnel with services, parades, and fireworks. A national moment of remembrance takes place at 3 p.m. At Arlington National Cemetery, headstones are graced with small American flags.

This day is not to be confused with Veterans Day, which is observed on November 11 to honor military veterans, both alive and dead.

However, confusion abounds anyway, with the weekend marking for many the kickoff of summer, and it is reserved for weekend getaways, picnics, and sales. Searches on "memorial day sales," "memorial day recipes," and "memorial day weekend" are just some of the lookups related to the festivities.

a.txt

23l4kj23 klgjdlskgj235 3lkj 0952ru lkfj lkqejfg
2t34lktj3409t uj34gjklejeglekjfdklsafjalsfj
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
sdgakdgjsdalgjaslfjsalkfjsadlf

b.txt

23l4kj23 klgjdlskgj235 3lkj 0952ru lkfj lkqejfg
2t34lktj3409t uj34gjklejeglekjfdklsafjalsfj
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
sdgakdgjsdalgjaslfjsalkfjsadlf

測試代碼如下：

ruby -EISO-8859-14 txtcmp.rb 1.txt 2.txt
1.txt and 2.txt semblance is 8.653846153846153%

ruby txtcmp.rb a.txt b.txt

a.txt and b.txt semblance is 79.54545454545455%

因為1.txt中包含非utf-8字元，預設比較會出錯，遂指定外部編碼比較。

ruby寫一個檔案內容相似性比較的代碼