D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 20149 - [DIP1000] Local data escapes inout method if not decorated with return
Summary: [DIP1000] Local data escapes inout method if not decorated with return
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: dmd (show other issues)
Version: D2
Hardware: All All
: P1 critical
Assignee: No Owner
URL:
Keywords: accepts-invalid, pull, safe
Depends on:
Blocks:
 
Reported: 2019-08-22 05:21 UTC by Mike Franklin
Modified: 2022-02-25 10:01 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Mike Franklin 2019-08-22 05:21:16 UTC
---
import std.stdio;

@safe:

struct ScopeBuffer(T, size_t Len)
{
    this(T[Len] buf, size_t len = 0)
    {
        this.buf = buf;
		this.len = len;
    }
    
    // Decorateing `opSlice` with `return`, causes the compiler to correctly emit
    // a compiler error on line 37, but without `return` a compiler error should be
    // emitted on line 25
    inout(T)[] opSlice(size_t lower, size_t upper) inout /*return*/
    in
    {
        assert(lower <= len, "Lower bound must be less than or equal to the length");
        assert(upper <= len, "Upper bound must be less than or equal to the length");
        assert(lower <= upper, "Lower bound must be less than or equal to the upper bound");
    }
    do
    {
        return buf[lower .. upper]; //BUG: compiler error shoud be emitted here
                                    // if `opSlice` is not decorated with `return`
    }
    
    T[Len] buf;
    size_t len;
}

char[] fun()
{
    char[4] buf = "abcd";
    auto sb = ScopeBuffer!(char, 4)(buf, 4);
    return sb[0..2];  // BUG:  compiler allows data internal to `ScopeBuffer` to 
                      // escape here unless `ScopeBuffer.opSlize` is decorated with `return`
}

void main()
{
    auto s = fun();
    writeln(s);
}
---
https://run.dlang.io/is/rNvdwC

Observation-1:  Code compiles with `dmd -preview=dip1000 source.d`
Expectation:  A compiler error should be emitted in `opSlice` where the slice of `ScopeBuffer`'s `buf` members is escaping.

Observation-2:  Decorating `opSlice` with `return` causes a compiler error to be correctly emitted where the slice is escaping `foo`, but without `return` a compiler error should be emitted where the slice is being returned from `opSlice`.
Comment 1 Mike Franklin 2019-08-22 05:42:28 UTC
> Observation-2:  Decorating `opSlice` with `return` causes a compiler error to be correctly emitted where the slice is escaping `foo`

In the example above, that should be `fun`, not `foo`.
Comment 2 Mike Franklin 2019-08-22 05:48:34 UTC
Interestingly removing the `inout` attribution on `opSlice` causes the compiler to emit an error at the return statement in `fun`.  I think the error should instead be emitted at the return statement of `opSlice`, but at least it doesn't compile.
Comment 3 Mike Franklin 2019-08-22 05:52:15 UTC
Removing all templating from the example above cause the compiler to emit what I think is correct behavior:

---
import std.stdio;

@safe:

struct ScopeBuffer
{
    this(char[4] buf, size_t len = 0)
    {
        this.buf = buf;
		this.len = len;
    }
    
    char[] opSlice(size_t lower, size_t upper)
    in
    {
        assert(lower <= len, "Lower bound must be less than or equal to the length");
        assert(upper <= len, "Upper bound must be less than or equal to the length");
        assert(lower <= upper, "Lower bound must be less than or equal to the upper bound");
    }
    do
    {
        return buf[lower .. upper]; // GOOD: Compiler error here
    }
    
    char[4] buf;
    size_t len;
}

char[] fun()
{
    char[4] buf = "abcd";
    auto sb = ScopeBuffer(buf, 4);
    return sb[0..2];
}

void main()
{
    auto s = fun();
    writeln(s);
}
---
https://run.dlang.io/is/Fo1CfP
Comment 4 Mike Franklin 2019-08-22 05:53:44 UTC
> Removing all templating from the example above cause the compiler to emit what I think is correct behavior:

That may mean this is a duplicate of Issue 19965, but I'm not sure.
Comment 5 Mike Franklin 2019-09-04 13:15:29 UTC
A more reduced test case:

---
import std.stdio;

@safe:

struct ScopeBuffer
{
    this(char[4] init)
    {
        this.buf = init;
    }
    
    // The bug is that `inout` implies `return` on `this`.
    // See https://github.com/dlang/dmd/blob/2dc5b15d1949148d460e8c809dd878e6dec8dfa3/src/dmd/func.d#L511-L513
    inout(char)[] opSlice(size_t lower, size_t upper) inout
    do
    {
        return buf[lower .. upper]; //BUG: compiler error should be emitted here
    }
    
    char[4] buf;
}

char[] fun()
{
    char[4] buf = "abcd";
    auto sb = ScopeBuffer(buf);
    return sb[0..2];
}

void main()
{
    auto s = fun();
    writeln(s);
}
---
https://run.dlang.io/is/Pu1125

The problem is that the compiler is inferring `return` on `this` when decorated with `inout`.

This was discussed in issue 17927.
Comment 6 Dlang Bot 2019-09-05 00:23:45 UTC
@JinShil created dlang/dmd pull request #10390 "Fix Issue 20149 - [DIP1000] Local data escapes `inout opSlice` if not annotated with `return`" fixing this issue:

- Fix Issue 20149 - [DIP1000] Local data escapes `inout opSlice` if not annotated with `return`

https://github.com/dlang/dmd/pull/10390
Comment 7 Dlang Bot 2019-09-05 00:41:30 UTC
@JinShil created dlang/druntime pull request #2785 "Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149" mentioning this issue:

- Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

https://github.com/dlang/druntime/pull/2785
Comment 8 Dlang Bot 2019-09-05 04:58:44 UTC
dlang/druntime pull request #2785 "Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149" was merged into stable:

- 4f1568cf77085dd3ca9c5e0468c22382fcc28a0d by Mike:
  Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

https://github.com/dlang/druntime/pull/2785
Comment 9 Walter Bright 2019-09-07 20:44:00 UTC
Not sure yet where the error is, but it isn't with the `inout`.
Comment 10 Walter Bright 2019-09-08 20:37:04 UTC
Here's what's actually happening. It has nothing to do with inout. A reduced example:
-----------------
@safe:

struct S {
  this(int) { }
  char[] opSlice() return { return buf[]; }
  char[4] buf;
}

S bar();

char[] fun() {
  return S()[]; // Error: escaping ref to stack value returned by S('\xff')
  return S(1)[]; // should generate same error here
  return bar()[]; // Error: escaping ref to stack value returned by bar()
}
----------------

The trouble is that the temporary returned by the constructor call is being treated differently than the temporary returned by the struct literal and the temporary returned by the function call, when they should be treated the same.
Comment 11 Dlang Bot 2019-09-09 01:52:46 UTC
@JinShil created dlang/dmd pull request #10402 "Issue 20149 - [DIP1000] temp returned from constructor call not checked for scope problems" mentioning this issue:

- Issue 20149 - [DIP1000] temp returned from constructor call not checked for scope problems

https://github.com/dlang/dmd/pull/10402
Comment 12 Mike Franklin 2019-09-09 01:54:20 UTC
(In reply to Walter Bright from comment #10)
> A reduced example:

https://github.com/dlang/dmd/pull/10402 should fix that issue, but unfortunately, it doesn't fix the original issue.
Comment 13 Dlang Bot 2019-09-09 04:23:51 UTC
dlang/dmd pull request #10402 "Fix Issue 20149 - [DIP1000] temp returned from constructor call not checked for scope problems" was merged into stable:

- c2ef8221d980d6c5957a77eae1ace8a17b3d3242 by Mike:
  Fix Issue 20149 - [DIP1000] temp returned from constructor call not checked for scope problems

https://github.com/dlang/dmd/pull/10402
Comment 14 Mike Franklin 2019-09-09 04:56:35 UTC
Reopening. The original issue still remains.
Comment 15 Walter Bright 2019-09-09 09:07:29 UTC
(In reply to Mike Franklin from comment #14)
> Reopening. The original issue still remains.

Please pare the example code down to the minimum. In fact, the original issue was fixed. If there is another issue, please leave this one closed, and open a new issue for the other issue.

I.e. one issue == one fix
Comment 16 Mike Franklin 2019-09-09 09:20:35 UTC
(In reply to Walter Bright from comment #15)
> Please pare the example code down to the minimum.

This is the most minimal example I know how to make to illustrate the issue:

---
@safe:

struct ScopeBuffer
{
    this(char[4] init)
    {
        this.buf = init;
    }

    inout(char)[] opSlice(size_t lower, size_t upper) inout
    {
        return buf[lower .. upper];
    }

    char[4] buf;
}

char[] fun()
{
    char[4] buf = "abcd";
    auto sb = ScopeBuffer(buf);
    return sb[0..2];
}

void main()
{
    auto s = fun();
}
---


> In fact, the original
> issue was fixed. If there is another issue, please leave this one closed,
> and open a new issue for the other issue.
> 
> I.e. one issue == one fix

The example above is the original issue and is related to how `inout` infers `return`.

The issue you added was a new, unrelated, issue related to how constructors are checked for scope issues.  I fixed that issue in 
https://github.com/dlang/dmd/pull/10402.

If we wanted one issue == one fix, we should have made a new issue for the check on constructors, but we're past that now.

The original issue remains, and I've submitted a fix to https://github.com/dlang/dmd/pull/10390
Comment 17 Dlang Bot 2019-09-20 14:24:21 UTC
@rainers updated dlang/druntime pull request #2805 "fix Issue 20227 - "Error: phread_mutex_destroy failed." after fork()" mentioning this issue:

- Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

- Merge pull request #2785 from JinShil/fix_20149
  
  Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

https://github.com/dlang/druntime/pull/2805
Comment 18 Dlang Bot 2019-09-27 07:28:24 UTC
@rainers updated dlang/druntime pull request #2802 "Issue 20219 - Idle D programs keep consuming CPU in Gcx.scanBackground" mentioning this issue:

- Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

- Merge pull request #2785 from JinShil/fix_20149
  
  Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

https://github.com/dlang/druntime/pull/2802
Comment 19 Dlang Bot 2019-10-03 06:21:56 UTC
@ikod updated dlang/druntime pull request #2811 "fix issue 20256" mentioning this issue:

- Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

- Merge pull request #2785 from JinShil/fix_20149
  
  Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

https://github.com/dlang/druntime/pull/2811
Comment 20 Dlang Bot 2019-10-03 20:08:55 UTC
@MartinNowak created dlang/dmd pull request #10447 "Merge remote-tracking branch 'upstream/stable' into merge_stable" fixing this issue:

- Fix Issue 20149 - [DIP1000] temp returned from constructor call not checked for scope problems

- Merge pull request #10402 from JinShil/fix_20149_2
  
  Fix Issue 20149 - [DIP1000] temp returned from constructor call not checked for scope problems

https://github.com/dlang/dmd/pull/10447
Comment 21 Dlang Bot 2019-10-03 20:09:10 UTC
@MartinNowak updated dlang/druntime pull request #2758 "Merge remote-tracking branch 'upstream/stable' into merge_stable" mentioning this issue:

- Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

- Merge pull request #2785 from JinShil/fix_20149
  
  Add `return` to `DSO.moduleGroup()` in support of compiler fix for issue 20149

https://github.com/dlang/druntime/pull/2758
Comment 22 Walter Bright 2020-03-15 08:32:20 UTC
(In reply to Mike Franklin from comment #16)
> (In reply to Walter Bright from comment #15)
> > Please pare the example code down to the minimum.
> 
> This is the most minimal example I know how to make to illustrate the issue:

Thank you. But:

1. how are you compiling it?
2. what do you expect to happen compared with what does happen?
(I compile it and no errors are emitted)
Comment 23 Rainer Schuetze 2020-03-15 11:06:34 UTC
(In reply to Walter Bright from comment #22)
> 1. how are you compiling it?

I guess with -preview=dip1000

> 2. what do you expect to happen compared with what does happen?
> (I compile it and no errors are emitted)

This is the described problem. It should produce an error, a pointer to stack allocated memory is escaping.
Comment 24 Dlang Bot 2021-06-15 23:33:04 UTC
@dkorpel created dlang/dmd pull request #12689 "Fix issue 22027, 20149 - inout doesn't imply return" fixing this issue:

- fix issue 22027, 20149 - inout doesn't imply return

https://github.com/dlang/dmd/pull/12689
Comment 25 Walter Bright 2022-02-14 09:42:37 UTC
A further simplified test case:

  @safe:

  struct S {
  //inout(char)* slice() inout return /* gives correct error */
    inout(char)* slice() inout          /* no error */
    {
        return &buf;
    }

    char buf;
  }

  char* fun() {
    S sb;
    return sb.slice();
  }

Note that if slice() were made as a static function, with sb passed by ref as the first parameter, the compilation gives the correct error. Hence it's related to the `this` being a special case.
Comment 26 Dlang Bot 2022-02-15 08:55:40 UTC
@WalterBright updated dlang/dmd pull request #13658 "fix Issue 20149 - [DIP1000] Local data escapes inout method if not de…" fixing this issue:

- fix Issue 20149 - [DIP1000] Local data escapes inout method if not decorated with return

https://github.com/dlang/dmd/pull/13658
Comment 27 Dlang Bot 2022-02-25 10:01:32 UTC
dlang/dmd pull request #13658 "fix Issue 20149 - [DIP1000] Local data escapes inout method if not de…" was merged into master:

- e0b6ea806a3a3bd0f1f4dff4d6662d5286b86889 by Walter Bright:
  fix Issue 20149 - [DIP1000] Local data escapes inout method if not decorated with return

https://github.com/dlang/dmd/pull/13658