Talk:Hamming code/Archive 1

Archive 1

(7,4)?

Maury, I like your rewrite a lot, but one thing jumps out immediately at me: isn't the Hamming code you describe a (7,4) rather than a (7,3)? You indicate earlier that Hamming's nomenclature classifies codes as ( total number of bits in a word , number of those bits that are data ). Well, if the powers of two are our parity bits, that's 1, 2 and 4, right? Leaving four data bits, at 3, 5, 6, and 7? -- Antaeus Feldspar 04:01, 30 Sep 2004 (UTC)

3, 5, 6, 7 <- 4 bits that are data ;-) -- KTC 00:00, 24 February 2005 (UTC)

KTC, that is exactly what I wrote. If you look at the article as it was at the time I wrote the above comment, you'll see that it described the standard Hamming code, the one that's meant if you just say "Hamming code", as a (7,3) code. It has since been changed to (7,4). I am not sure what you are trying to indicate. -- Antaeus Feldspar 01:25, 24 Feb 2005 (UTC)

My bad. I didn't read properly and thought you meant shouldn't (7,4) be (7,3). Ignore me ;-D -- KTC 23:01, 24 Feb 2005 (UTC)

Incorrect $H_{e}$ matrix

I think the He matrix in (7,4) example is incorrect. It's last three lines should be:
0 1 1 1
1 0 1 1
1 1 0 1

Zurlich 13:27, 15 January 2006 (UTC)

I think you're right. Good catch. I've (hopefully) fixed it, now. Explanation: if you xor the rows in $H_{e}$ as selected by each line (in turn) from $H_{d}$, you should get [0 0 0 0] each time. I.e. for the middle row of $H_{d}$, you are looking at rows 2, 3, 6 and 7 from $H_{e}$. It's probably possible to fix the example by changing $H_{d}$, instead. StuartBrady 14:45, 15 January 2006 (UTC)

Incorrect $H_{d}$ matrix?

Looking at the $H_{d}$ matrix:

$H_{d}:={\begin{pmatrix}0&0&0&1&1&1&1\\0&1&1&0&0&1&1\\1&0&1&0&1&0&1\\\end{pmatrix}}$

The top row corresponds to bits 4-7. The middle row corresponds to bits 2-3 and 6-7. The bottom row corresponds to bits 1, 3, 5 and 7. So should the top and bottom rows be swapped? StuartBrady 15:22, 15 January 2006 (UTC)

Real Hamming Matrices?

Does the author of the (7,4) code example have a reference? It appears to contradict Hamming's 1950 paper, as well as the other example on the page.

Hamming placed the data bits in positions 3, 5, 6, 7. But this matrix places them in positions 1, 2, 3, 4. Is this still considered a "Hamming code" in modern terminiology? To match Hamming's original description, the matrix should have the rows in a different order:

$H_{e}:={\begin{pmatrix}1&1&0&1\\1&0&1&1\\1&0&0&0\\0&1&1&1\\0&1&0&0\\0&0&1&0\\0&0&0&1\\\end{pmatrix}}$

If both versions are commonly accepted, then there needs to be an explanation to this effect in the article. Scottcraig 07:33, 18 February 2006 (UTC)

Teletext places data bits in 2, 4, 6, 8. The literature still refers to it as a Hamming code (even though it uses an extra parity bit). The ordering of the rows isn't important, so long as $H_{e}$ and $H_{d}$ are consistent. Note that may be better to describe the positions of the check bits when referring to Hamming's original description, which I expect are 1, 2, 4, 8, 16, ...

However, I found the whole article confusing. The explanation I was given is effectively (in my own words) "Place check bits at bit positions (with 1 being the least significant bit) 1, 2, 4, 8, 16, etc. Initially, set the check bits to 0. Then XOR together the indices of all bits that are set (counting from 1), and place the resulting value in the check bits. When receiving, repeat the XORing operation. If the result is 0, there we no detectable errors. Otherwise, if the error was a 1-bit error, the result is the index of the bit that was corrupted. If more bits were corrupted, there is no way to determine which to correct." The matrix representation just generalises this slightly. I'm not sure how/whether to add that to the article, though. Obviously there would need to be references and examples. --StuartBrady 15:00, 18 February 2006 (UTC)

I don't find the whole article confusing, just a little bare. Your explanation of Hamming codes here is basically the same as that in the (11,7) example. But it could evidently be improved given that you find it confusing.

My problem is with the (7,4) example. The author first states that the modern usage of "Hamming code" is restricted to the (7,4) code given in the 1950 paper. But then he immediately provides an example of a different code, along with "Hamming matrices" which aren't in the paper either.

Actually, Hamming illustrates his codes using the (7,4) code, but describes codes of arbitrary sizes as well. He also relates his work in the theory of error-correcting codes, most notably his concept of (Hamming) distance between code words. I think his (7,4) code was actually used at Bell Labs, but I could use some confirmation on that. In his paper, he also defines a version with have an extra parity bit, which allows for detection of 2-bit errors besides correction of 1-bit errors. (So maybe teletext is justly called a Hamming code.)

In the course I'm taking right now, we learned about "linear codes" of which Hamming codes and cyclic codes are special cases. There appears to be a very short and cryptic page on these already. Maybe we should reorganise the material among these two pages. The historical Hamming codes could belong on this page, and the matrix theory could go on the other.

In any case, the (7,4) example needs revamping at least, if not removal. The (11,4) example could use a better explanation, and maybe be modified to be of (7,4) size to match the historical description. Any other opinions? Scottcraig 21:19, 18 February 2006 (UTC)

I'm concerned that you don't consider it confusing — note that I didn't say "totally unreadable". Sure, my explanation is basically the same as the (11,7) example, but the example has little explanation. I think the author of the (7,4) example is probably wrong to state the modern usage of the "Hamming code" is retricted to the (7,4) code, but I'm not sure. Using the version in Hamming's paper can't hurt. I didn't realise that the extra parity bit was in Hamming's paper — thanks for pointing that out. Given that it took me 20 or so minutes to verify a correction to a simple mistake in the matrix, perhaps the explanation could be improved. --StuartBrady 03:29, 19 February 2006 (UTC)

Note on difference between (7,4) and (11,7) descriptions

I've noted in the article that the used matrices use a different order for data and parity bits. This at least should clear some misunderstanding.

Blaisorblade 15:53, 14 June 2006 (UTC)

In the example using 11,7 code, the table shows a parity bit calculation for the entire word including the error-flipped bit. The bit is shown as a 1 but the parity of the word itself is a 0. Am I missing something? [removed my IP address] kendallcp 13:10, 1 October 2006 (UTC)

Well spotted. I've removed it. --StuartBrady (Talk) 14:10, 1 October 2006 (UTC)

New images

I created a slew of SVGs for the (7,4) example. Please check my work and let me know if I messed something up as I'll be happy to fix it (or fix it yourself if you've got an SVG editor (e.g., Inkscape)). Cburnett 21:23, 31 December 2006 (UTC)

(Pretty sure this should say four here instead of three, right?)

Hi 66.90.203.197, you're right that the text wasn't quite clear. Indeed if at most 3 errors are made, you will always see that an error was made. But there is an qualitative difference between them. For the (4,1) repetition code we have the following situations:

• At most 1 error is made. Any error is detected and corrected correctly.
• At most 2 errors are made. Any error is detected but may not be corrected
• At most 3 errors are made. Any error is detected but may be corrected to the wrong code word.
• At most 4 errors are made. Some errors will not be detected any more.

So when you flip 3 bits that is the first time you will start to receive messages that after decoding turn out to be wrong, while you can't see this.

For this reason, the code really is only useful if you have only 1 error. With 3 errors it is completely useless. —The preceding unsigned comment was added by Sander123 (talk • contribs) 08:38, 12 March 2007 (UTC).

---

The Hamming code has a minimum distance of 3, which means detect/correct 1 error. Add ing the extra partiy bit, extends the minimum distance to 4, which means correct/detect 1 error, detect 2 errors, however cannot correct for those 2 errors.

So to try to analyse what happens when you introduce 3 or more errors, does not make sense.

In general for any block-linear code, of which the Hamming Code is one, the error correction properties of the code for dealing with random errrors can be summarised by the minimum distance of the code.

The minimum distance is defined as the minimum hamming distance between two codewords (after the partity bits have been added). This calculation is performed across all codewords. I am sure there is a mathematical proof.

Anyway I have changed the text in that section.

Hi sorry I'd like to ask a question

When you work out your Hamming Code, why do you add parity bits on the left? If the first Parity bit, P1, is supposed to be on the first bit, should it not be on the right? Hence, with your (11,7) example, where you have _ _ 0 _ 1 1 0 _ 1 0 1, shouldn't you have 0 1 1 _ 0 1 0 _ 1 _ _? Giving you a Hamming code of... 0 1 1 0 0 1 0 1 1 1 0, using Even parity.

That's what I read in my lecture notes, anyway. And I spot alot of different "versions" on the internet, so I'd like to just seek your opinion. Thanks LordDazzer 10:08, 4 April 2007 (UTC)

Bit order is irrelevant so long as it is understood. :) It's an age-old argument of endianness. Cburnett 12:51, 4 April 2007 (UTC)

The placement of parity bits is important to exploit the properties of the code.
The placement of parity and data in the transmitted sequence should be
p0 p1 p2 t3 p4 t5 t6 t7 p8 t9 t10 t11 t12 t13 t14 t15 p16 t17 t18 etc
Notice I label the parity bits P0, p1, p2, p4, p8, etc, the suffix represents in the transmission stream where
the parity bits should be located. And t3=d0, t5=d1, t6=d2, t7=d3, t9=d4, t10=d5 etc is how the input sequence
is mapped.

On reception when you calculate the parity bits,
If all p0-pn are zero, no errors occured
If p0=0, and p1-pn non zero, the decimal value of p1-pn points to the single error
If p0=1, then a double error occured

pn is calculated using the skip n-1, then (check n, skip n) procedure described in the main text
p0 is the even parity across the whole transmitted sequence

Hamming (8,4) parity check matrix wrong

Im no expert, but i think the table is missing a column. Other sources seem to agree and not to mention its supposed to be a code of length 8! --Ray andrew (talk) 16:04, 13 December 2007 (UTC)

You're right. I've fixed this now. At some point, I'm going to convert these to the more standard systematic form. Oli Filth^(talk) 17:51, 13 December 2007 (UTC)

That used in (7,4) is systemic. (8,4), as shown in the diagrams, is identical to (7,4) with a parity bit covering the whole thing. The matrices weren't of the correct dimensions but as they were a minute ago then were 100% inconsistent with the diagrams. I have restored my previous version with dimension fixed. Please show some math if I've got it wrong. Cburnett (talk) 02:20, 8 March 2008 (UTC)

You are correct; whilst my correction (13 Dec 07) fixed the dimensionality, I didn't notice that it didn't match the diagram.

You are also correct that the codes are both systematic, in that all the data bits appear in the codewords. By systematic, I guess I meant the more aesthetically pleasing arrangement such that the data bits form the first (or last) k bits of the codeword (typical of e.g. cyclic codes), such that the generator matrix may be expressed in the form

$\ G={\begin{bmatrix}I&G^{\prime }\end{bmatrix}}$

Oli Filth^(talk) 12:01, 8 March 2008 (UTC)

Link back to chanel coding?

This is a form of channel coding, so should this link back to there? - Mohammed Petiwala(MADDY)

test —Preceding unsigned comment added by 121.52.51.50 (talk) 05:45, 8 July 2008 (UTC)

What does "2-out-of-5" have to do with error correction?

The 2/5 article says nothing about it. Neither does this one. So why is it discussed here? Seems BOTH articles need expansion, OR, that THIS article needs citation because this appears dubious. 76.254.85.70 (talk) 03:37, 13 August 2009 (UTC)

Missing example?

The second paragraph under "Hamming codes" refers to a (3,1) repetition code example, but there is no such example in the article. —Preceding unsigned comment added by Mreftel (talk • contribs) 09:18, 9 September 2009 (UTC)

Obtuse explanation for generating the hamming bits.

It's amazing how an article that started out with a clear and concise algorithm for generating the hamming bits once again gets obfuscated by attempts to over-formalize the mathematical explanation. These methods are rarely used in hardware implementations. —Preceding unsigned comment added by 76.77.231.233 (talk) 23:32, 21 August 2010 (UTC)

Broken references to "the repetition example" / the (3,1) code

The repetition example has not been part of this page for two years, yet it is referred to several times. If it's a useful example, we should add it back. Personally, I think it's useful, and I intend to re-add it (and delete this comment) when I get some free time. Piojo (talk) 02:16, 25 April 2010 (UTC)

I noticed this problem also just now, and have searched through the edit history. It was removed by an IP editor in December 2008. The deletion may have been undetected vandalism. I have accordingly copied the last version of the 12/2010 "Repetition" section, and re-instated it. I know very little about this subject, and I have made no attempt to ensure that the restored section is consistent with the rest of the changes since 12/2008, but I do think it is a useful and important section to have, especially as it is referenced in the current article. Wwheaton (talk) 00:14, 1 November 2010 (UTC)

Other common codes

Are there any other common codes besides (7,4) and (11,7)? Cburnett 07:41, 2 January 2007 (UTC)

yes there are a family of codes

7,4
15,11
31,26
63,57
...
2^N - 1, 2^N - (N+1)

And each code can be appended with the parity bit to make the minimum distance 4

I appreciate the response but I asked for common ones. :) Are all the ones you listed common in use? Cburnett 13:11, 5 April 2007 (UTC)

I don't know which implementations are the most common, but the family given above is used because the total number of bits is close to powers of two, which are often required for I/O operations (32bit words, for instance). Within the constraints of I/O, you would want to use the largest code possible, since that will have the highest information rate (approaching one as N goes to infinity). Quantum7 04:55, 12 June 2007 (UTC)

And of course, there's (3,1), when N=2 --208.80.119.67 (talk) 04:00, 9 November 2010 (UTC)

Mistake in intro??

If the Hamming distance were less than two, then a single error could be undetectable because it could convert between legal bit patterns.

Shouldnt it be uncorrectable???nishantjr (talk) 16:49, 16 December 2008 (UTC)

Nope. If the Hamming distance is less than two, then it is one. A Hamming distance of 1 means that valid patterns are adjacent (differ by 1 bit). --208.80.119.67 (talk) 04:03, 9 November 2010 (UTC)

General Algorithm Section Suggestions

Could you add a definition for the term "covers". Also,

"To check for errors, check all of the parity bits. The pattern of errors, called the error syndrome, identifies the bit in error. If all parity bits are correct, there is no error. Otherwise, the sum of the positions of the erroneous parity bits identifies the erroneous bit. For example, if the parity bits in positions 1, 2 and 8 indicate an error, then bit 1+2+8=11 is in error. If only one parity bit indicates an error, the parity bit itself is in error."

This seems to need an example based entirely on the general algorithm. The linked article explaining "error syndrome" depends on the understanding of the generator and parity-check matrices. The generator and parity-check matrices are explained only later in this article and in the linked article Hamming (7,4) which refers back to this section. In both cases, the reader's understanding is dependent on understanding this section.

In particular, illustrating the creation the generator matrix used later for the case of 4 bit decoded-word to 7 bit coded-word directly from the algorithm would help.

Mloiselle (talk) 00:52, 18 March 2011 (UTC)

(7,4) split

I split the (7,4) stuff to Hamming(7,4). Cburnett 03:13, 2 January 2007 (UTC)

I rewrote the entire article (Hamming(7,4)) to follow the general Hamming code generation. Please review to make sure I got everything right. Cburnett 06:23, 2 January 2007 (UTC)

---

In section "Construction of G and H" in: $\mathbf {G} :={\begin{pmatrix}I_{k}|-A^{T}\\\end{pmatrix}}$

What do $I_{k}$ and $A^{T}$ stand for? In the context of this page it is not clear, so they should be eiter referenced by a link or explained. (I would have explained them if I knew :) ) —Preceding unsigned comment added by Tanbaum (talk • contribs) 13:58, 20 May 2011 (UTC)

Reed-Solomon connection?

How do Hamming Codes relate to Reed-Solomon error correction? Why does ECC memory use a Hamming code but a CD use Reed-Solomon? Inquiring minds want to know... —Preceding unsigned comment added by 67.183.110.101 (talk) 07:21, 28 February 2011 (UTC)

Hamming codes are used in DRAM ECC because encoding and decoding for Hamming code is fast. Reed Solomon is used in CDs because the distance in Reed Solomon can be arbitrary and so can correct large number of errors but the distance in Hamming codes are limited (3). CDs can have scratches which result in sometimes lots of errors which is different to DRAM memory error characteristics. Of course, the Reed Solomon codes are slower but CDs don't have to run as fast as DRAM.

I'm not sure about the relationship. Any code with the right dimensions and distance 3 is permutationally equivalent to the Hamming code so a Reed Solomon code of that has that dimension is a permutationally equivalent Hamming code. — Preceding unsigned comment added by Mochan Shrestha (talk • contribs) 05:39, 28 October 2011 (UTC)

Hamming Distance

Isn't the distance of Hamming codes always 3? For binary or q-ary? Why say $d$ as the distance and give properties that apply to more than Hamming codes? Mochan Shrestha (talk) 05:54, 28 October 2011 (UTC)

The Hamming(7,4)-code has distance 3, but the family of Hamming codes defined in this article can have larger distance. ylloh (talk) 17:04, 28 October 2011 (UTC)

Yes, but the larger distance ones are extended Hamming codes and not strictly Hamming codes. Even so considering the extended Hamming code, $d=3$ or $d=4$ (extended) for Hamming codes but the article is very general about the distance. Every reference I've seen has Hamming codes as $\left[{\frac {q^{r}-1}{q-1}},{\frac {q^{r}-1}{q-1}}-r,3\right]$ code (written as $[n,k,d]$). So, the article should maybe be modified to say the $d$ is only takes the value 3 or 4? Mochan Shrestha (talk) 21:48, 28 October 2011 (UTC)

Yes, you're right! The generalized Hamming codes that I was thinking of have distance 3 (eg., see [1]). I edited the article accordingly. ylloh (talk) 16:02, 29 October 2011 (UTC)

(11,7) is not a Hamming Code

I hestitate to remove the section since it has been there so long and contains so much detail, but surely this just isn't a Hamming Code.

A Hamming Code has length 2^n - 1, so there's no Hamming Code of length 11.

Also, the (8,4) code is, strictly, and extended Hamming Code.

Before I delete the (11,7) code, would someone like to dispute/support what what I say?Daveofthenewcity (talk) 10:20, 9 July 2010 (UTC)

No one said anything, so I have now deleted it. It is shame to lose so much, but it was misleading to suggest that this is a Hamming Code. Daveofthenewcity (talk) 15:15, 16 July 2010 (UTC)

A Hamming code can have length 10 if it's a non-binary code. The length of a non-binary code is ${\frac {q^{r}-1}{q-1}}$ and for $q=9$ and $r=2$, we have a [10,7] code. Then, possibly an extension to distance 4 to get the [11,7] code? Mochan Shrestha (talk) 06:08, 28 October 2011 (UTC)

Mochan Shrestha, that is true, but the particular deleted section we are discussing discusses a binary code.

Daveofthenewcity, Yes, that section had some really nice illustrations, so I am sad to see it go.

I agree that it is technically not a Hamming code, and I don't have any evidence that it is notable enough or educational enough to justify including it here.

But that immediately leads me to ask -- if it's not a Hamming code, then what is it? Does it have some other name, and we can move those nice illustrations to the Wikipedia article with that name?

The article currently briefly mentions "the (72,64) code, a truncated (127,120) Hamming code".

That (72,64) code is widely used in Dimm#Error correction, and therefore notable enough to be described somewhere in the encyclopedia.

That (11,7) code could be formed by starting with a Hamming(15,11) code and truncating in the same way -- dropping a few data bits, but keeping all the check bits.

Is there any name more specific for that particular (11,7) code than "a (11,7) code, a truncated Hamming(15,11) code", and is it actually used or notable in any way? --DavidCary (talk) 17:00, 25 February 2012 (UTC)

Hamming(7,4) matrices G and H

These are different in this article and in Hamming(7,4). Is at least one of these correct? Matma Rex pl.wiki talk 23:07, 24 March 2012 (UTC)

They both are. The generator matrix can have any order of its columns (note that the two generators have the same column entries, just different orderings) as long as they contain the k-columns of the k-by-k identity matrix and the (n-k) columns of the (n-k)-by-(n-k) inverted identity matrix for the parity bits. We note in the article: "Finally, these matrices can be mutated into equivalent non-systematic codes by the following operations [Moon, p. 85]: Column permutations (swapping columns) ...". The advantage of using the code on this page (as opposed to the one listed on Hamming(7,4) is that this code is systematic. In other words, the k message bits are carried into the n-bit block unchanged. jheiv ^{talk contribs} 17:53, 28 May 2012 (UTC)

Parity Bit not including itself

I think there is a small error in general algorithm description. In Section "Hamming Codes", subsection "General algorithm" under point 5 it is stated that each parity bit covers even the parity bit itself. I think that is wrong. Parity bit 1 only covers 3,5,7, etc. It should not be 1,3,5,7, etc. The same is for the other parity bits.

In German Wikipedia the formula in [2] is correct. — Preceding unsigned comment added by MartinGarbe (talk • contribs) 10:48, 16 December 2013 (UTC)

Error

http://picpaste.com/Screenshot_from_2014-01-19_17_11_45.png Wrong Infobox TheKing44 (talk) 22:17, 19 January 2014 (UTC)

Error in section "Hamming(7,4) code"?

1. The text says: "So G can be obtained from H by taking the transpose of the left hand side of H with the identity k-identity matrix on the left hand side of G."

The left hand side of H is the matrix:

${\begin{pmatrix}1&1&0&1\\1&0&1&1\\0&1&1&1\end{pmatrix}}$

The transponse matrix is:

${\begin{pmatrix}1&1&0\\1&0&1\\0&1&1\\1&1&1\end{pmatrix}}$

The right side of matrix G is:

${\begin{pmatrix}1&1&0\\0&1&1\\1&0&1\\1&1&1\end{pmatrix}}$

Maybe this is correct because the text says that the columns can be swapped.

2. The two matrices H and G must satisfy $\mathbf {H} \,\mathbf {G} ^{T}=\mathbf {0}$. Using the two given matrices of the example i get:

${H}\cdot {G}^{T}={\begin{pmatrix}1&1&0&1&1&0&0\\1&0&1&1&0&1&0\\0&1&1&1&0&0&1\end{pmatrix}}\cdot {\begin{pmatrix}1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1\\1&0&1&1\\1&1&0&1\\0&1&1&1\end{pmatrix}}={\begin{pmatrix}0&1&1&0\\0&1&1&0\\0&0&0&0\end{pmatrix}}\neq {\begin{pmatrix}0&0&0&0\\0&0&0&0\\0&0&0&0\end{pmatrix}}$

Your matrix after "The right side of matrix G" does not appear on the page currently, so probably the error has been fixed (although I couldn't find the matrix you give in the history either). 130.233.158.198 (talk) 15:39, 18 August 2014 (UTC)

This G^T is not correct. When it is fixed, the computation works. Bill Cherowitzo (talk) 03:36, 19 August 2014 (UTC)

Dual Hamming codes

The dual of the Hamming code is actually the shortened Hadamard code and not the punctured one, since we obtain the dual Hamming code by simply throwing away the leading zero from all the codewords in the Hadamard code of appropriate size. — Preceding unsigned comment added by 2602:30A:C088:2050:112E:D1FB:A828:FAF4 (talk) 00:46, 12 November 2016 (UTC)