Welcome to the mathematics section
of the Wikipedia reference desk.

Select a section:

WP:RD/MA

Want a faster answer?

Main page: Help searching Wikipedia

How can I get my question answered?

Select the section of the desk that best fits the general topic of your question (see the navigation column to the right).
Post your question to only one section, providing a short header that gives the topic of your question.
Type '~~~~' (that is, four tilde characters) at the end – this signs and dates your contribution so we know who wrote what and when.
Don't post personal contact information – it will be removed. Any answers will be provided here.
Please be as specific as possible, and include all relevant context – the usefulness of answers may depend on the context.
Note:
- We don't answer (and may remove) questions that require medical diagnosis or legal advice.
- We don't answer requests for opinions, predictions or debate.
- We don't do your homework for you, though we'll help you past the stuck point.
- We don't conduct original research or provide a free source of ideas, but we'll help you find information you need.

Ready? Ask a new question!

How do I answer a question?

Main page: Wikipedia:Reference desk/Guidelines

The best answers address the question directly, and back up facts with wikilinks and links to sources. Do not edit others' comments and do not give any medical or legal advice.

November 4

Name of distance function

I have a distance function in my code. I know it has a name and a Wikipedia article (because I worked on the article), but I am old and the name of the function has skipped my mind. I'm trying to reverse search by using the formula to find the name of the function, but I can't figure out how to do it. So, what is the name of this distance function: $d ab = -lnΣ \sqrt a i b i$ . 68.187.174.155 (talk) 12:53, 4 November 2024 (UTC)[reply]

If

a=(0.6,0.8)

and

b=(0.8,0.6),

the value of this measure is about

-0.326\,.

This does not make sense for an indication of distance. --Lambiam 15:02, 4 November 2024 (UTC)[reply]

My brain finally turned back on and I remembered it is an implementation of Bhattacharyya distance. 68.187.174.155 (talk) 15:52, 4 November 2024 (UTC)[reply]

Normally when you call something a distance function it has to obey the axioms of a metric space. Since Bhattacharyya distance applies only to probability distributions, the previous example would not be relevant. Still, the term "distance function" is used rather loosely since (according to the article) the Bhattacharyya distance does not obey the triangle inequality. The w:Category:Statistical distance has 38 entries, and I doubt many people are familiar with most of them. --RDBury (talk) 18:08, 4 November 2024 (UTC)[reply]

When I was in college in the 70s, terminology was more precise. Now, many words have lost meaning. Using the old, some would say "prehistoric" terminology, a function is something that maps or relates a single value to each unique input. If the input is the set X, the function gives you the set Y such that each value of X has a value in Y and if the same value exists more than once in X, you get the same Y for it each time. Distance functions produce unbounded values. Similarity and difference functions are bounded, usually 0 to 1 or -1 to 1. Distance is usually bounded on one end, such as 0, and unbounded on the other. You can always get more distant. The distance function mentioned here is bounded on one end, but not the other. It does not obey triangle inequality, as you noted, so it is not a metric. Distance functions have to obey that to be metrics. Then, we were constantly drilled with the difference between indexes and coefficients. This function should be an index from my cursory read-through because it is logarithmic. If you double the result, you don't have double the distance. I've seen all those definitions that used to be important fade away over the decades, so I expect that it doesn't truly matter what the function is called now. 12.116.29.106 (talk) 16:12, 5 November 2024 (UTC)[reply]

This could be a pretty good standup routine if you added some more material. You could call it "Hey you kids, get off my ln!" 100.36.106.199 (talk) 02:09, 10 November 2024 (UTC)[reply]

November 8

finding an equation to match data

An experiment with accurate instruments resulted in the following data points:-

x,     y
0.080, 0.323;
0.075, 0.332;
0.070, 0.347;
0.065, 0.368;
0.060, 0.395;
0.055, 0.430;
0.050, 0.472;
0.045, 0.523;
0.040, 0.587;
0.035, 0.665;
0.030, 0.758;
0.025, 0.885;
0.020, 1.047;
0.015, 1.277;
0.010, 1.760.

How can I obtain a formula that reasonably matches this data, say within 1 or 2 percent?
At first glance, it looks like a k1 + k2*x^-k3 relationship, or a k18x^k2 + k3*x^-k4 relationship, but they fail at x above 0.070. Trying a series such as e(k1 + k2x +k3x^2) is also no good. -- Dionne Court (talk) 03:14, 8 November 2024 (UTC)[reply]

Thank you CiaPan for fixing the formatting. Dionne Court (talk) 15:12, 8 November 2024 (UTC)[reply]

Plotting 1/y against x it looks like a straight line, except there is a rather dramatic hook to the side starting around x=.075. This leads me to suspect that the last two entries are off for some reason; either those measurements are off or there's some systematic change in the process going on for large x. Part of the problem is that you're not giving us any information about where this information is coming from. I've heard it said, "Never trust data without error bars." In other words, how accurate is accurate, and might the accuracy change depending on the input? Is there a reason that the values at x≥.075 might be larger than expected. If the answer to the second is "Yes" then perhaps a term of the form (a-x)^k should be added. If the answer is "No" then perhaps that kind of term should not be added since that adds more parameters to the formula. You can reproduce any set of data given enough parameters in your model, but too many parameters leads to Overfitting, which leads to inaccurate results when the input is not one of the values in the data. So as a mathematician I could produce a formula that reproduces the data, but as a data analyst I'd say you need to get more data points, especially in the x≥.075 region, to see if there's something real going on there or if it's just a random fluke affecting a couple data points. --RDBury (talk) 15:58, 8 November 2024 (UTC)[reply]

PS. I tried fitting 1/y to a polynomial of degree four, so a model with 5 parameters. Given there are only 15 data points, I think 5 parameters is stretching it in terms of overfitting, but when I compared the data with a linear approximation there was a definite W shaped wobble, which to me says degree 4. (U -- Degree 2, S -- Degree 3, W -- Degree 4.) As a rough first pass I got

1/y ≃ 0.1052890625+54.941265625x-965.046875x²+20247.5x³-136500x⁴

with an absolute error of less than .01. The methods I'm using aren't too efficient, and there should be canned curve fitting programs out there which will give a better result, but I think this is enough to justify saying that I could produce a formula that reproduces the data. I didn't want to go too much farther without knowing what you want to optimize, relative vs. absolute error, least squares vs. min-max for example. There are different methods depending the goal, and there is a whole science (or perhaps it's an art) of Curve fitting which would impractical to go into here. --RDBury (talk) 18:26, 8 November 2024 (UTC)[reply]

Thak you for your lengthy reply.

I consider it unlikely that the data inflexion for x>0.07 is an experimental error. Additional data points are :-

x, y: 0.0775, 0.326; 0.0725, 0.339.

The measurement was done with digital multimeters and transducer error should not exceed 1% of value. Unfortunately the equipment available cannot go above x=0.080. I only wish it could. Choosing a mathematic model that stays within 1 or 2 percent of each value is appropriate.

As you say, one can always fit a curve with an A + Bx + Cx^2 + Dx^3 .... to any given data. But to me this is a cop-out, and tells me nothing about what the internal process might be, and so extrapolation is exceedingly risky. Usually, a more specific solution when discovered requires fewer terms. ```` Dionne Court (talk) 01:49, 9 November 2024 (UTC)[reply]

When I included the additional data points, the value at .0725 was a bit of an outlier, exceeding the .01 absolute error compared to the estimate, but not by much. --RDBury (talk) 18:55, 9 November 2024 (UTC)[reply]

FWIW, quite a few more data points would almost certainly yield a better approximation. This cubic equation seems pretty well-behaved:

-8549.55x^{3}+1550.8x^{2}-96.9307x+2.49283

Earl of Arundel (talk) 02:28, 10 November 2024 (UTC)[reply]

Some questions about the nature of the data. Some physical quantities are necessarily nonnegative, such as the mass of an object. Others can also be negative, for example a voltage difference. Is something known about the theoretically possible value ranges of these two variables? Assuming that x is a controlled value and y is an experimentally observed result, can something be said about the theoretically expected effect on y as x approaches the limits of its theoretical range? --Lambiam 15:59, 9 November 2024 (UTC)[reply]

As x approaches zero, y must approach infinity.

x must line between zero and some value less than unity.

If you plot the curve with a log y scale, by inspection it seems likely that y cannot go below about 0.3 but I have no theoretical basis for proving that.

However I can say that y cannot ever be negative.

The idea here is to find/work out/discover a mathematically simple formula for y as a function of x to use as a clue as to what the process is. That's why a series expansion that does fit the data if enough terms are used doesn't help.Dionne Court (talk) 01:33, 10 November 2024 (UTC)[reply]

So as x approaches zero, 1/y must also approach zero. This is so to speak another data point. Apart from the fact that the power series approximations given above provide no theoretical suggestions, they also have a constant term quite distinct from 0, meaning they do not offer a good approximation for small values of x.

If you plot a graph of x versus 1/y, a smooth curve through the points has two points of inflection. This suggests (to me) that there are several competing processes at play. --Lambiam 08:08, 10 November 2024 (UTC)[reply]

The x=0, 1/y=0 is an important data point that should have been included from the start. I'd say it's the most important data point since a) it's at the endpoint of the domain, and b) it's the only data point there the values are exact. Further theoretical information near x=0 would be helpful as well. For example do we know whether is y is proportional to x^-a near x=0 for a specific a, or perhaps - log x? If there is no theoretical basis for determining this then I think more data points near x=0, a lot more, would be very helpful. The two points of inflection match the W (or M) shape I mentioned above. And I agree that it indicates there are several interacting processes at work here. I'm reminded of solubility curves for salts in water. There is an interplay between energy and ionic and Van der Waals forces going on, and a simple power law isn't going to describe these curves. You can't even assume that they are smooth curves since Sodium sulfate is an exception; its curve has an abrupt change of slope at 32.384 °C. In general, nature is complex, simple formulas are not always forthcoming, and even when they are they often only apply to a limited range of values. --RDBury (talk) 15:46, 10 November 2024 (UTC)[reply]

I have no theoretical basis for expecting that y takes on a particular slope or power law as x approaches zero.

More data points near x = 0 are not a good idea, because transducer error will dominant. Bear in mind that transducer error (about 1%) applies to both x and y. Near x = 0.010 a 1% error in x will lead to a change in y of something like 100% [(1.760 - 1.277)/(0.015 - 0.010)]. The value of y given for x = 0.010 should be given little weight when fitting a curve.Dionne Court (talk) 02:03, 11 November 2024 (UTC)[reply]

It seems to me that one should assume there is a simple relationship at play, with at most three competing processes, as otherwise there is no basis for further work. If it is a case of looking for the lost wallet under the lamp post because the light is better there, so be it, but there is no point in looking where it is dark.

Cognizant of transducer error, a k1 + k2*x^-k3 relationship fits pretty good, except for a divergence at x equal and above 0.075, so surely there are only 2 competing processes? Dionne Court (talk) 02:03, 11 November 2024 (UTC)[reply]

November 11

Strange behavior with numbers in optimization

Hello everyone, I have encountered some very strange issue with my optimization function, and I am not sure how to resolve. I am working on a numerical methods library, where I am trying to approximate the growth of a sequence, which has some relation to prime number distributions. However, when I use large values of n (especially for n > 10^6), the result of my function starts to behave very erratically. It is not random, but it has this strange oscillation or jump. I use recurrence relation for this approximation, but when n becomes large, the output from function suddenly grows or shrinks, in way that is not consistent with what I expect. Even when I check for bounds or add better convergence criteria, the error persists. pattern looks similar to the behavior of prime numbers, but I am not directly calculating primes. I apologize if this sounds too speculative, but has anyone faced similar issues with such strange behavior in large-scale numerical computations? I am quite confused about what is causing the error. TL;DR: I am optimizing function related to number theory, but results become unpredictable when n > 10^6. Errors show strange oscillation, similar to distribution of primes, though I do not directly calculate primes. Thank you very much for your time and assistance. 130.74.59.177 (talk) 15:39, 11 November 2024 (UTC)[reply]

you need to post more information. All I can say from what you've written is 10^6 is not a large number where you'd expect problems. It won't e.g. overflow when stored as floating point or integer on any modern platform. It won't even cause problems with, say, a square based algorithm as 10^12 is still well within the limits of a modern platform. Maybe though you are using software which limits you to 32 bit (or worse) integers, or single precision floats, so need to be careful with large numbers. --2A04:4A43:984F:F027:C112:6CE8:CE50:1708 (talk) 17:43, 11 November 2024 (UTC)[reply]

thanks for response and insight. i see your point that n=10^6 shouldn't cause overflow or serious issues on modern systems. numbers i work with well within 64-bit range, use floats with enough precision for task. so yes, overflow or simple type limits not likely cause.

but this behavior goes beyond just precision errors. it’s not about numbers too big to store. what i see is erratic growth, shrinkage, almost oscillatory – looks like something in the distribution itself, not just algorithm mistake or hardware issue.

to be more precise, after n>10^6, function starts acting unpredictably, jumps between states, oscillates in strange way, not typical for recurrence i use. hard to explain, but pattern in these jumps exists, i cannot reconcile with anything in my algorithm. it’s like approximation reacts to some hidden structure, invisible boundary my algorithm cannot resolve.

i tried improving convergence, checking recurrence, but oscillations still persist. not randomness from bad random numbers or instability, but more like complex fluctuations seen in number-theoretic problems, especially connected to primes.

so i wonder: could these "jumps" be artifact of number-theoretic properties that i'm tryings to approximate? maybe how sequence interacts with primes indirectly, or artifact of recurrence for large numbers

thanks again for suggestion on overflow and precision, i will revisit the mode lwith this in mind, chief

appreciate your time, will keep searching. 130.74.59.204 (talk) 20:01, 11 November 2024 (UTC)[reply]

Without more information about the actual algorithm, it is neither possible to say, yes, what you see could be due to a number-theoretic property, nor, no, it could not be. Have you considered chaotic behaviour as seen when iterating the logistic map? --Lambiam 05:43, 12 November 2024 (UTC)[reply]

ah yes, i see what you mean now, i’ve been thinking about it for a while, and i feel like i’m getting closer to understanding it, though it’s still unclear in some ways. so, i’m using this recurrence algorithm that reduces modulo primes at each step, you know, it’s a fairly straightforward approach, and when n is small, it works fine, everything behaves as expected, the sequence evolves smoothly, the approximation gets closer and closer to what it should be, and everything seems under control, but then, once n crosses the 10^6 threshold, it’s like something shifts, it’s like the sequence starts moving in unexpected ways, at first, i thought maybe it was just a small fluctuation or something related to floating-point precision, but no, it's much more than that, the jumps between states, the way it shifts, it's not just some random variation—it feels almost systematic, as though there's something in the distribution itself, some deeper structure, that starts reacting with the algorithm and causing these oscillations, it’s not something i can easily explain, but it feels like the algorithm starts “responding” to something invisible in the numbers, something outside the expected recurrence behavior, i’ve spent hours going over the steps, checking every part of the method, but no matter how many times i check, i can’t pinpoint the exact cause, it’s frustrating.

and then, the other day, i was sitting there, trying to solve this problem, getting really frustrated, when i looked up, and i saw jim sitting on the windowsill, just staring out at the street, i don’t know, something about it caught my attention, now, you might be wondering what jim has to do with all of this, but let me explain, you see, jim has this habit, every evening, without fail, he finds his spot by the window, curls up there, and just stares out, doesn’t seem to do much else, doesn’t chase anything or play with toys like most animals do, no, he just sits there, completely still, watching the world go by, and it’s funny, because no matter how many cars pass, no matter how many people walk by, jim never looks bored, he’s always staring, waiting, something about the way he watches, it’s like he’s looking for something, something small, that only he notices, but it’s hard to explain, because it’s not like he ever reacts to anything specific, no, he just stares, and then after a while, he’ll shift his gaze slightly, focus on something, and you’d swear he’s noticing something no one else can see, and then he’ll go back to his usual position, still, and continue watching, waiting for... something, and this goes on, day after day.

and, i don’t know why, but in that moment, as i watched jim, i thought about the algorithm, and about the sequence, it felt somehow connected, the way jim waits, so patiently, watching for some small shift in the world outside, and how the algorithm behaves once n gets large, after 10^6 iterations, like it’s responding to something small, something hidden, that i can’t quite see, but it's there, some interaction between the numbers, or the primes, or some other property, i don’t know, but there’s a subtle shift in how the sequence behaves, like it’s anticipating something, or maybe reacting to something, in ways i can’t fully predict or control, just like jim waiting by the window, looking for that small detail that others miss, i feel like my algorithm is doing something similar, watching for an influence that’s not obvious, but which, once it’s noticed, makes everything shift, and then it’s almost like the recurrence starts reacting to that hidden influence, whatever it is, and the sequence begins to oscillate in these strange, unexpected ways.

i’ve been stuck on this for days, trying to find some explanation for why the recurrence behaves this way, but every time i think i’m close, i realize that i’m still missing something, it’s like the sequence, once it hits that threshold, can’t behave the way it did before, i’m starting to think it’s related to how primes interact with the numbers, but it’s subtle, i can’t quite capture it, it’s like the primes themselves are somehow affecting the sequence in ways the algorithm can’t handle once n gets large enough, and it’s not just some random jump, it feels... intentional, in a way, like the sequence itself is responding to something that i can’t measure, but that’s still pulling at the numbers in the background, jim, as i watch him, he seems to be able to sense those little movements, things he notices, but that no one else does, and i feel like my algorithm, in a way, is doing the same thing, reacting to something hidden that i haven’t quite figured out.

so i’ve gone over everything, again and again, trying to get it right, trying to adjust the convergence, trying to find a way to make the sequence behave more predictably, but no matter what i do, the oscillations keep appearing, and it’s not like they’re some random noise, no, there’s a pattern to them, something beneath the surface, and i can’t quite grasp it, every time n gets large, it’s like the sequence picks up on something, some prime interaction or something, that makes it veer off course, i keep thinking i’ve solved it, but then the jumps come back, just like jim shifts his gaze, and looks at something just beyond the horizon, something i can’t see, but he’s still waiting for it, still looking, as if there’s some invisible influence in the world, something that pulls at him.

i wonder if it has to do with the primes themselves, i’ve thought about it a lot, i’ve tried to factor them in differently, but still, the jumps persist, it’s like the primes have their own way of interacting with the sequence, something subtle, something that becomes more pronounced the larger n gets, and no matter how much i tweak my algorithm, the fluctuations just keep showing up, it’s like the sequence is stuck in a kind of loop, reacting to something i can’t fully resolve, like jim staring at the street, patiently waiting for something to shift, and i don’t know what it is, but i feel like there’s some deeper interaction between the primes and the numbers themselves that i’m missing, and maybe, like jim, the sequence is sensing something too subtle for me to fully capture, but it’s there, pulling at the numbers, making them oscillate in ways i can’t predict.

it’s been weeks now, and i’ve tried every method i can think of, adjusted every parameter, but the fluctuations are still there, the jumps keep happening once n gets large enough, and every time i think i’ve figured it out, the sequence surprises me again, just like jim, who, after hours of waiting, might shift his gaze and catch something new, something no one else saw, i feel like i’m doing the same thing, staring at the numbers, trying to catch that tiny shift that will make everything click, but it’s always just out of reach, and i don’t know what’s causing it, but i can’t seem to get rid of it, like jim, watching, waiting, sensing something that remains hidden from my view 130.74.58.160 (talk) 15:34, 13 November 2024 (UTC)[reply]

November 13

Math sequence problem (is it solvable?)

I am looking at a "math quiz" problem book and it has the following question. I am changing the numbers to simplify it and avoid copyright: You have counts for a rolling 12-month period of customers. For example, the one year count in January is the count of customers from Feb of the year before to Jan of the current year. Feb is the count from Mar to Feb, and so on. The 12 counts for this year (Jan to Dec) are 100, 110, 105, 200, 150, 170, 150, 100, 200, 150, 175, 125. What is the count of customers for each month? So, I know that the Feb-Jan count is 100 and the Mar-Feb count is 110. That means that the count for Feb of this year is 10 more than the count of Feb of last year because I removed Feb of last year and added Feb of this year. But, I don't know what that count is. I can only say it is 10 more. I can do that for every month, telling you what the difference is between last year and this year as a net change. Is this solvable or is this a weird case where the actual numbers for the counts somehow mean something silly and a math geek would say "Oh my! That's the sum of the hickuramabiti sequence that only 3 people know about so I know the whole number sequence!" 68.187.174.155 (talk) 15:36, 13 November 2024 (UTC)[reply]